Senior Software Engineer - Data (License Team)

Who We Are

• At Sonatype, we help organizations build better, more secure software by enabling them to understand and control their software supply chains. Our products are trusted by thousands of engineering teams globally, providing critical insights into dependency health, license risk, and software security. We’re passionate about empowering developers—and we back it with data.

The Opportunity

• We’re looking for a Senior Data Engineer with full stack expertise to join our growing Data Platform team.
• This role blends data engineering, microservices, and fullstack development to deliver end-to-end services that power analytics, machine learning, and advanced search across Sonatype.
• You will design and build data-driven microservices and workflows using Java, Python, and Spring Batch, implement frontends for data workflows, and deploy everything through CI/CD pipelines into AWS ECS/Fargate.
• You’ll also ensure services are monitorable, debuggable, and reliable at scale, while clearly documenting designs with Mermaid-based sequence and dataflow diagrams.
• This is a hands-on engineering role for someone who thrives at the intersection of data systems, fullstack development, ML, and cloud-native platforms.

What You’ll Do

• Design, build, and maintain data pipelines, ETL/ELT workflows, and scalable microservices.
• Development of complex web scraping (playwright) and realtime pipelines (Kafka/Queues/Flink)Develop end-to-end microservices with backend (Java 5+, Python 5+, Spring Batch 2+) and frontend (React or any).
• Deploy, publish, and operate services in AWS ECS/Fargate using CI/CD pipelines (Jenkins, GitOps).
• Architect and optimize data storage models in SQL (MySQL, PostgreSQL) and NoSQL stores.
• Implement web scraping and external data ingestion pipelines.
• Enable Databricks and PySpark-based workflows for large-scale analytics.Build advanced data search capabilities (fuzzy matching, vector similarity search, semantic retrieval).
• Apply ML techniques (scikit-learn, classification algorithms, predictive modeling) to data-driven solutions.Implement observability, debugging, monitoring, and alerting for deployed services.Create Mermaid sequence diagrams, flowcharts, and dataflow diagrams to document system architecture and workflows.
• Drive best practices in fullstack data service development, including architecture, testing, and documentation.

What We’re Looking For

• 5+ years of experience as a Data Engineer or a Software Backend engineering role.
• Strong programming skills in Python, Scala, or JavaHands-on experience with HBase or similar NoSQL columnar stores.
• Hands-on experience with distributed data systems like Spark, Kafka, or Flink.
• Proficient in writing complex SQL and optimizing queries for performance.
• Experience building and maintaining robust ETL/ELT pipelines in production.
• Familiarity with workflow orchestration tools (Airflow, Dagster, or similar).
• Understanding of data modeling techniques (star schema, dimensional modeling, etc.).
• Familiarity with CI/CD pipelines (Jenkins or similar).
• Ability to visualize and communicate architectures using Mermaid diagrams.
• Bonus Points
• Experience working with Databricks, dbt, Terraform, or Kubernetes.
• Familiarity with streaming data pipelines or real-time processing.
• Exposure to data governance frameworks and tools.
• Experience supporting data products or ML pipelines in production.
• Strong understanding of data privacy, security, and compliance best practices