background

Senior Cloud Developer/Data Engineer

Job Description

We are looking for an experienced and innovative Senior Cloud Developer/Data Engineer to join our team in Hyderabad. The ideal candidate will have a strong background in designing and developing data pipelines using Scala or Python, with significant experience in AWS technologies. You will be responsible for enhancing large-scale applications, creating robust data processing pipelines, and solving complex data integrations across multiple systems. Your ability to provide technical guidance, collaborate with business units, and drive real-time data analysis strategies will be crucial in delivering high-quality solutions. If you are passionate about leveraging cutting-edge technologies and enjoy working in a dynamic and fast-paced environment, we want to hear from you.

Why Join Us

  • Opportunity to work with cutting-edge technologies and innovative solutions.
  • Collaborate with a dynamic team in a fast-paced environment.
  • Engage with top-tier clients and contribute to impactful projects.
  • Competitive compensation and benefits package.

Responsibilities

  • Collaborate as part of a development team to design and enhance large-scale applications using Python and Spark.
  • Evaluate and plan software designs, test results, and technical manuals using AWS.
  • Confer with business units and development staff to understand both business and technical requirements for producing technical solutions.
  • Create and review technical and user-focused documentation for data solutions (data models, data dictionaries, business glossaries, process and data flows, architecture diagrams, etc.).
  • Extend and enhance the business Data Lake. Create or implement solutions for metadata management.
  • Solve complex data integrations across multiple systems.
  • Design and execute strategies for real-time data analysis and decision-making.
  • Build robust data processing pipelines using AWS Services and integrate with multiple data sources.
  • Translate client user requirements into data flows and data mappings.
  • Analyze and determine data integration needs and follow Agile practices.

Responsibilities

  • Collaborate as part of a development team to design and enhance large-scale applications using Python and Spark.
  • Evaluate and plan software designs, test results, and technical manuals using AWS.
  • Confer with business units and development staff to understand both business and technical requirements for producing technical solutions.
  • Create and review technical and user-focused documentation for data solutions (data models, data dictionaries, business glossaries, process and data flows, architecture diagrams, etc.).
  • Extend and enhance the business Data Lake. Create or implement solutions for metadata management.
  • Solve complex data integrations across multiple systems.
  • Design and execute strategies for real-time data analysis and decision-making.
  • Build robust data processing pipelines using AWS Services and integrate with multiple data sources.
  • Translate client user requirements into data flows and data mappings.
  • Analyze and determine data integration needs and follow Agile practices.

Required Experience

  • At least 5+ years of experience in designing and developing Data Pipelines for Data Ingestion or Transformation using Scala or Python.
  • At least 4 years of experience with Python and Spark.
  • At least 3 years of experience working with AWS technologies.
  • Experience in designing, building, and deploying production-level data pipelines using tools from AWS Glue, Lambda, Kinesis, and databases like Aurora and Redshift.
  • Experience with Spark programming (Pyspark or Scala).
  • Hands-on experience with AWS components like EMR, S3, Redshift, Lambda, API Gateway, and Kinesis in production environments.
  • Strong analytical skills and advanced SQL knowledge, including indexing and query optimization techniques.
  • Experience using ETL tools for data ingestion.
  • Experience with Change Data Capture (CDC) technologies and relational databases such as MS SQL, Oracle, and DB.
  • Ability to translate data needs into detailed functional and technical designs for development, testing, and implementation.
  • 5+ years of Development / Data Engineering Experience.
  • 4+ years developing AWS Cloud solutions.
  • 4+ years developing with Spark/Pyspark.
  • 4+ years developing AWS Glue ETL.
  • 4+ years AWS Storage models (S3 and DynamoDB).
  • Some hands-on experience with on-prem ETL tooling (Ab Initio preferred, Informatica).

Desired Skills

  • Strong experience in AWS Glue, EMR, and Hudi, including extracting data from multiple sources and loading data into Data Lakes AWS Redshift.
  • Experience working with AWS Elasticsearch, RDS, and PostgreSQL preferred.
  • Experience with AWS services such as Lambda, EMR, SNS/SQS, EventBridge, Lake Formation, and Athena.
  • Experience integrating applications/systems (data producers) with Enterprise Kafka topics (Confluent Kafka integration with AWS S3, Redshift).
  • Experience in Requirements Analysis, Data Analysis, Application Design, Application Development, and Integration Testing.
  • Working knowledge of on-prem Extraction, Transformation, Cleansing, and Loading methodology and principles.
  • Experience implementing and adding to DevOps principles (GitLab, maintaining CI/CD pipelines).
  • Experience with Service Oriented Architecture (SOA).

Qualifications

  • Any Bachelor’s Degree

Location

  • Hyderabad, Telangana

Job Type

  • Full Time

How to Apply

  • Interested candidates are invited to submit their resume and cover letter outlining their qualifications and experience to hr@rythmos.com with the subject line "Senior Cloud Developer Application – [Your Name]".
  • Our offices are located in Dallas, Hyderabad, and Coimbatore, India.
  • Rythmos is an equal opportunity employer that considers all applications for employment without regard to race, sex, age, color, religion, national origin, veteran status, disability, sexual orientation, gender identity, genetic information, or any characteristic protected by law.