Are you looking for an opportunity to work in a new supply chain product? We are a startup team working to enable organizations across the world with reliable, cost effective and flexible end-to-end supply chain solutions, to help them scale, succeed and offer best in class experience to their customers.
As an Amazon Data Engineer you will be working in one of the world's largest and most complex data warehouse environments. Our team is responsible for timely delivery of mission critical analytical reports and metrics that are viewed at the highest levels in the organization. You should have deep expertise in the data ingestion pipeline design, creation, management and business use of large datasets. You should have excellent business and communication skills to be able to work with business owners to develop and define key business questions, and to build data sets that answer those questions. You should be expert at designing, implementing, and operating stable, scalable, low cost solutions to flow data from production systems into the data warehouse and into end-user facing applications. You should be able to work with business customers in a fast paced environment understanding the business requirements and implementing reporting solutions. Above all you should be passionate about working with huge data sets and someone who loves to bring datasets together to answer business questions and drive change.
Key job responsibilities
- Work with SDE teams and business stakeholders to understand data requirements and design data ingress flow for team
- Lead the design, model, and implementation of large, evolving, structured, semi-structured and unstructured datasets
- Evaluate and implement efficient distributed storage and query techniques
- Interact and integrate with internal and external teams and systems to extract, transform, and load data from a wide variety of sources
- Implement robust and maintainable code with clear and maintained documentation
- Implement test automation on code implemented through unit testing and integration testing
- Work in a tech stack which is a mix of NAWS services and legacy ETL tools within Amazon
About the team
Data Insights, Metrics & Reporting team (DIMR) is the central data engineering team in Amazon Warehousing & Distribution org which is responsible for 4 main things-
1. Building and maintaining entire analytics-reporting infrastructure
2. Building data ingestions pipelines from any kind of ingress
3. Building mechanisms to vend data to internal team members or external sellers with right data handling techniques in place
4. Build insights generation tools and frameworks using latest Gen AI technologies
- 3+ years of data engineering experience
- Experience with data modeling, warehousing and building ETL pipelines
- Experience with AWS technologies like Redshift, S3, AWS Glue, EMR, Kinesis, FireHose, Lambda, and IAM roles and permissions
- Experience with non-relational databases / data stores (object storage, document or key-value stores, graph databases, column-family databases)
- Experience with Airflow, Spark, ETL tools, Data Lake, Data Warehouse, Data Modelling