We are looking for a BigData developer to implement 35-300 billions records AWS based storage for genotypes data, Data Access Layer API to be used by 10+ applications.
The project is aiming for storing and serving genotype information which is currently handled by two different existing systems.
The project aims to promote data availability to support multiple key R&D processes in genotype data production, data aggregations, data imputations, hapmap construction, breeding decisions, and assay design by having centralized access to all investments in genotyping data (assay - internal & external genotyping technologies and sequence based – NGS & GBS).
Project’s major design principles:
· provide a unified API for Genotypes, decreasing the impact of internal code changes for downstream application and making genotypes easy to consume;
· scalable in terms of data volume and allowing multiple loading processes simultaneously;
· extensible to minimize the amount of development effort needed for implementing new business functionalities.
- Help in evolving data analytics, data warehouse, ETL, and big data capabilities to achieve the best possible solutions for our business and applications
- Integrate, improve, productize POCs
- Develop REST services
- Implement imports/exports/data copying from upstream applications
- Partner with the business and development teams to deliver data solutions and respond to support and enhancement inquires
- Collaborate with our data modeler and data architects, to help improve our data delivery, security and reporting capabilities for the enterprise
- Work with distributed team
- Big Data, AWS, Java are the primarily one
- Spring, REST, Spring Boot, Spark
- Performance optimization experience
- English B1/B1+Minimum
- 3+ years of experience in design, development, deployment and support activities using Enterprise ETL platforms or other data integration tools or processes
Nice to have
- Athena, EMR, Airflow, Scoop
- Competitive compensation depending on experience and skills
- Individual career path in engineering
- Social package - medical insurance, sports
- Compensation for sick lists and regular vacations
- English classes with native speakers (certified English teachers)
- Flexible work hours