Title: Pyspark Developer

Primary Skill: Python

Secondary Skill: SQL, Azure

Seniority: Senior

Years: 7

Emp Type: Contract

Location: Tri-State, NJ

Client position: 562

Description:

Our client is seeking a skilled data processing specialist to design and develop innovative solutions on a leading cloud platform using a popular Python-based data processing framework. The goal is to leverage extensive experience in this framework and hands-on expertise in cloud computing to navigate complex data challenges, ensuring adherence to industry regulations while driving meaningful outcomes. Our client aims to rapidly adapt to emerging technologies and create scalable data solutions that enhance operational efficiency and informed decision-making.

Responsibilities:

• Develop, maintain, and optimize data pipelines using the aforementioned Python-based framework on the cloud platform.
• Collaborate with cross-functional teams to understand data requirements and translate business needs into technical specifications.
• Design and implement scalable data models and architecture to support analytical and reporting needs.
• Perform data refinement, transformation, and enrichment processes to enhance data quality and usability.
• Monitor and troubleshoot data processing jobs, ensuring high availability and performance.
• Implement best practices for data governance and security within the cloud platform.
• Stay up-to-date with emerging trends in data management, cloud computing, and the client's industry.

Requirements:

• Minimum of 9 years of experience in developing data processing solutions using the Python-based framework.
• Proven experience working with the cloud platform's services (including data integration, data engineering, and analytics tools).
• Prior experience in a related sector is mandatory.
• Proficient in the Python-based framework for data processing and ETL tasks.
• Experience with relational and non-relational databases.
• Familiarity with data storage concepts and tools.
• Knowledge of cloud services related to data management and analytics.

Preferred, but not required:

• Experience with virtualization technologies (e.g., containerization, orchestration).
• Familiarity with predictive modeling frameworks (e.g., machine learning libraries, deep learning tools).

External URL external

Edit | Back (all JDs) | Back (client position)