Big Data Engineer - Hadoop & Spark

Join our dynamic team at Sonsoft Inc. as a Big Data Engineer specializing in Hadoop and Spark. In this role, you'll leverage cutting-edge technologies to process and analyze large datasets, providing actionable insights that drive business decisions. Ideal candidates are passionate about big data technologies and eager to tackle complex challenges in a fast-…

Join our dynamic team at Sonsoft Inc. as a Big Data Engineer specializing in Hadoop and Spark. We are seeking a passionate and experienced individual who thrives in a fast-paced environment and is eager to tackle complex data challenges. In this role, you will be responsible for designing, developing, and implementing scalable big data solutions that enhance business performance.

Join usm2 as a Senior Data Modeler / Data Architect with expertise in Big Data and Hadoop. In this pivotal role, you will harness the power of data to drive strategic decisions and enhance business outcomes. Your experience with data modeling and architecture will be essential in building robust data ecosystems.

Join our dynamic team at Integrated Resources Inc. as a Hadoop Engineer. In this role, you will be instrumental in developing and optimizing our big data solutions using Hadoop technologies. Your expertise in data processing and analytics will help drive our projects forward, making a tangible impact on our clients and the industry.We are looking for a dedicated engineer who can collaborate effectively with cross-functional teams, ensuring that our data infrastructure is robust and scalable. If you are passionate about data engineering and want to be part of a forward-thinking company, we encourage you to apply!

Join our dynamic team at Sonsoft Inc. as a Hadoop Developer. In this role, you will be responsible for designing and implementing robust data processing solutions using Hadoop technologies. You will collaborate closely with data engineers and analysts to optimize data workflows and ensure high data quality.Ideal candidates will have a passion for big data and experience in developing applications that utilize Hadoop ecosystem tools such as Hive, Pig, and HBase. You will be instrumental in transforming raw data into actionable insights that drive business decisions.

Join Sonsoft Inc. as a Big Data Architect in the vibrant city of San Francisco. We are looking for an innovative and experienced professional to lead our data architecture initiatives. In this role, you will design and implement scalable data solutions that empower our clients to make data-driven decisions.

We are seeking a talented Big Data Architect to join our dynamic team at Sonsoft Inc. in San Francisco. In this role, you will be responsible for designing and implementing robust data architectures that support our data-driven initiatives. You will work closely with cross-functional teams to ensure that our data architecture aligns with business goals and delivers high-quality data solutions.

Join Sonsoft Inc. as a Big Data Architect and play a pivotal role in shaping our data-driven solutions. We are seeking a talented professional who will design and implement robust data architectures that support our analytics and business objectives. You will work closely with cross-functional teams to build scalable data pipelines and ensure data integrity across our platforms.

Join our innovative team at Sonsoft Inc. as a Big Data Architect, where you will play a pivotal role in architecting and implementing cutting-edge data solutions. You will leverage your expertise in big data technologies to design scalable, high-performance data architectures that drive business intelligence and analytics. This is an exciting opportunity to work with state-of-the-art tools and technologies in a collaborative environment.

Join our dynamic team at Sonsoft Inc. as a Hadoop Developer in the vibrant city of San Francisco. In this role, you will leverage your expertise in big data technologies to design, develop, and maintain robust Hadoop solutions that drive our data strategy forward. You will collaborate closely with cross-functional teams to ensure data integrity and performance optimization.

Join our dynamic team at Sonsoft Inc. as a Cloud Analytics Specialist! In this role, you will leverage Hadoop technologies to drive data analytics and cloud solutions. Your expertise will help us transform data into actionable insights, driving business growth and innovation.

Azumo is in search of a dynamic Big Data Engineer to spearhead the development and enhancement of our data and analytics infrastructure. This role is fully remote and based in Latin America.As a member of our innovative team, you will collaborate with forward-thinking engineers in the field of big data computing. If you are passionate about designing and developing scalable, high-performance big data infrastructure leveraging technologies such as Spark, Kafka, Snowflake, or similar frameworks, both on-premise and in the cloud, this position is perfect for you. We are seeking candidates with experience in building data pipelines, data services, data warehouses, as well as BI and ML platforms.At Azumo, we are committed to excellence and believe in fostering both professional and personal growth. We strive for each individual’s success and are dedicated to helping our team achieve their goals during their tenure at Azumo and beyond. Embracing challenges and acquiring new technologies is at the core of our mission. We also value giving back to the community through philanthropy, open-source initiatives, and knowledge sharing.

Join our dynamic team at Sonsoft Inc., a leading technology consulting firm, as a Big Data Architect. In this pivotal role, you will design and implement scalable big data solutions that empower organizations to harness the power of data. Collaborate with cross-functional teams to drive innovation and deliver high-impact projects.

P-1011 Job Location: San Francisco Bay Area, CA As a Senior Staff Technical Solutions Engineer and a recognized technical expert, you will collaborate closely with our Field and Engineering teams to provide exceptional, specialized support and customized technical solutions for Databricks' most significant and strategic clients in the Digital Native Business (DNB) segment. In this client-facing position, you will apply your technical knowledge of Apache Spark™ and other data technologies to diagnose and resolve intricate product issues and help remove obstacles for our customers' most pressing technical challenges. The Impact You Will Have Conduct advanced troubleshooting and root cause analysis to resolve performance and reliability issues in Spark, SQL, Delta, Streaming, and Databricks runtime features using tools like Spark UI metrics, Mosaic AI Model Service, Directed Acyclic Graphs (DAGs), and event logs. Identify requirements for continuous monitoring to proactively detect performance issues while collaborating with R&D and NOC teams to optimize customer environments within the DNB segment. Create rapid Proof of Concepts (POCs), test, deploy, and monitor solutions developed by Databricks Engineering to tackle customer challenges and demonstrate advanced Spark/ML/AI runtime capabilities in alignment with their business objectives. Compile extensive playbooks and maintain a knowledge base of common issues and solutions regarding Spark, ML, and AI workflows. Educate customer engineering and business teams on best practices in performance tuning, debugging, and effectively utilizing Databricks Features. Pilot and advocate for new best practices, champion process enhancements, and collaborate with cross-functional teams to improve the overall customer experience. Act as an advocate for customers during business review meetings and maintain close relationships as a trusted advisor and primary technical point of contact. Work onsite with Field Engineering, Sales, and Product teams during customer interactions and technical presentations to deliver swift resolutions to production-impacting issues, showcasing deep technical expertise and fostering strong customer trust.

Join our innovative team at Sonsoft Inc. as a Cloud Analytics Specialist, where you will harness the power of Hadoop to drive data insights and analytics in a cloud environment. We are seeking a talented professional who is passionate about leveraging cloud technologies to transform data into actionable strategies.Your role will involve designing and implementing robust analytics solutions, utilizing Hadoop for big data processing, and collaborating with cross-functional teams to optimize our data infrastructure. If you are looking to make a significant impact and advance your career in a dynamic company, we want to hear from you!

About tvScientific tvScientific is the pioneering CTV advertising platform specifically designed for performance marketers. We harness extensive data and advanced technologies to automate and enhance TV advertising, ultimately driving measurable business results. Our platform seamlessly integrates media buying, optimization, measurement, and attribution into a single, efficient solution. Developed by industry veterans with deep expertise in programmatic advertising, digital media, and ad verification, our CTV performance platform offers advertisers a reliable avenue to expand their business. As a Senior Data Engineer at tvScientific, you will play a crucial role in establishing the robust data infrastructure that supports our data-intensive operations. You will work in collaboration with cross-functional teams to refine our core data pipelines, ensuring efficient scaling as we grow, and optimizing data storage solutions. This individual contributor role requires you to define and execute a strategic vision for data engineering within the company. Key Responsibilities: Develop and implement a robust data infrastructure in AWS, utilizing Spark with Scala. Enhance our core data pipelines to efficiently accommodate our significant growth. Optimize data storage solutions in appropriate engines and formats. Collaborate with cross-functional teams to design data solutions that align with business objectives. Construct fault-tolerant batch and streaming data pipelines. Leverage and optimize AWS resources while scaling design. Work closely with Data Science and Product teams to achieve collective goals. Success Metrics: Successful establishment of scalable and efficient data infrastructure. Timely delivery and optimization of data assets and APIs. High attention to detail in the implementation of automated data quality checks. Effective collaboration with cross-functional teams. What We’re Looking For: Proven experience in production data engineering. Expertise in Spark and Scala, with a track record of building data infrastructure using these technologies. Familiarity with data lakes, cloud warehouses, and various storage formats. Strong proficiency in AWS services. Advanced SQL skills for data manipulation and extraction. Exceptional written and verbal communication abilities. Bachelor's degree in Computer Science or a related discipline. Nice to have: Experience with additional data processing frameworks.

About AlembicAlembic is at the forefront of transforming marketing strategies, demonstrating the actual ROI of marketing initiatives. Our cutting-edge Alembic Marketing Intelligence Platform employs advanced algorithms and AI models to address this longstanding challenge effectively. By joining our team, you'll contribute to the development of tools that deliver unparalleled insights into how marketing influences revenue, empowering a growing roster of Fortune 500 companies to make data-driven decisions with confidence.About the RoleIn your role as a Senior Data Engineer at Alembic, you will play a crucial role in our data platform. You will be responsible for creating scalable and dependable data pipelines, optimizing storage solutions, and facilitating both real-time and batch analytics. Collaborating closely with data scientists, software engineers, and product leaders, you will design and implement robust data architectures that propel our mission forward.Key ResponsibilitiesDesign, develop, and maintain scalable ETL pipelines that efficiently ingest, process, and transform extensive volumes of structured and unstructured data.Optimize data storage solutions utilizing modern data lakehouse architectures and industry best practices to enhance cost-effectiveness, performance, and reliability.Collaborate with data scientists and engineers to seamlessly integrate machine learning models and analytical workloads into production environments.Ensure the integrity, quality, and security of data by implementing monitoring, alerting, and governance best practices.Work with cloud-based data warehouses and distributed data processing frameworks to support our data initiatives.Continuously assess and implement innovative technologies to enhance data infrastructure and operational efficiency.What We’re Looking For10+ years of experience in data engineering, software engineering, or a related field.Strong proficiency in SQL and Python for data processing.Experience with contemporary data warehousing and lakehouse solutions (e.g., Iceberg or similar).Expertise in distributed systems and big data technologies (Apache Spark, Hadoop, Kafka, Flink).Hands-on experience with cloud platforms (AWS, GCP, Azure) and related data services.Deep understanding of data management and governance practices.

Join our dynamic team at 360itprofessionals1 as a Java Lead specializing in Big Data technologies. In this pivotal role, you will lead a talented group of developers in the design, development, and implementation of innovative solutions that leverage Big Data frameworks. This position is ideal for an experienced professional looking to make a significant impact in a fast-paced environment.

Founded in 2007, Airbnb has transformed the way people experience travel, connecting over 5 million hosts with more than 2 billion guests worldwide. Our platform enables unique stays and authentic experiences, fostering connections with local communities.The Team You Will Join:As a pivotal member of the Data Warehouse Infrastructure team, you will help shape the backbone of Airbnb's big data capabilities, enabling hundreds of engineers to efficiently collect, manage, and analyze vast amounts of data. We leverage cutting-edge open-source technologies such as Hadoop, Spark, Trino, Iceberg, and Airflow.Typical Responsibilities:Design and architect Airbnb's next-generation big data compute platform to enhance data ETL, analytics, and machine learning efforts.Oversee the platform's operations, focusing on improving reliability, performance, observability, and cost-effectiveness.Create high-quality, maintainable, and self-documenting code while engaging actively in code review processes.Contribute to open-source projects, making a significant impact on the industry.

At Databricks, we are driven by a passion for empowering data teams to tackle the world’s most challenging problems — from transforming transportation to accelerating medical innovations. We achieve this by creating and maintaining the leading data and AI infrastructure platform, enabling our clients to leverage profound data insights for business enhancement. Founded by engineers with a customer-first mentality, we eagerly embrace every opportunity to tackle complex technical challenges, ranging from the design of next-generation UI/UX for data interactions to scaling our services across millions of virtual machines. Our journey has just begun.As a member of the Runtime team at Databricks, you will be instrumental in developing the next generation of distributed data storage and processing systems. These systems will surpass specialized SQL query engines in relational query performance while offering the programming abstractions necessary to support a variety of workloads, from ETL to data science.Example projects include:Apache Spark™: Contribute to the de facto open-source standard framework for big data.Data Plane Storage: Develop reliable and high-performance services and client libraries for managing vast amounts of data within cloud storage backends like AWS S3 and Azure Blob Store.Delta Lake: Design a storage management system that merges the scalability and cost-effectiveness of data lakes with the performance and reliability of data warehouses, providing features like ACID transactions and time travel.Delta Pipelines: Simplify the orchestration and operation of numerous data pipelines, enabling clients to deploy, test, and upgrade pipelines effortlessly.Performance Engineering: Create the next-generation query optimizer and execution engine that is fast, scalable, and robust.