Job Description
•Python Scripting: Adeptly script in Python to develop and optimize data processing systems.
•SQL Proficiency: Utilize SQL for querying, transformation, and data performance optimization.
•Apache Airflow: Experience with Directed Acyclic Graphs (DAGs) in Airflow or a similar tool for workflow scheduling and monitoring.
•Version Control: Expertise in version control, branching, and collaboration on GitHub.
•Collaboration: Translate business requirements into technical solutions by working closely with stakeholders.
•Communication: Strong spoken English skills for effective team communication and stakeholder engagement.
•Additional Technologies: Familiarity with Docker, Kubernetes, AWS/cloud platforms, and unit testing.
Job Requirements
Required Skills:
•Experience: Minimum of 8 years of relevant experience, demonstrating proficiency in the key activities and required skills.
•Python Expertise: Extensive experience in Python programming, with a strong understanding of its libraries and frameworks.
•Database Management: Proficient in SQL and experience with relational databases, ensuring efficient data querying and management.
•Workflow Automation: Proficient in using Apache Airflow or similar workflow automation tools for managing data pipelines.
•Version Control Systems: Advanced knowledge of Git and GitHub, including branching strategies and collaborative workflows.
•Cloud Platforms: Experience with cloud services, particularly AWS, for deploying and managing applications.
•Containerization and Orchestration: Familiarity with Docker and Kubernetes for building, deploying, and scaling applications.
•Software Development Practices: Strong grasp of unit testing, code reviews, and continuous integration/continuous deployment (CI/CD) practices.
•Data Processing: Experience in building and optimizing data processing systems for large-scale data.
Nice-to-have requirements:
•Elixir Programming: Experience with Elixir.
•Data Warehousing: Familiarity with Cloudera Data Warehouse.
•BI Tools: Experience with AWS Athena/Tableau.