What you’ll do:
Being a member of the data-team, you will be working in a cross-discipline, full stack delivery team focused on one of many core business objectives.
Gather and process raw data at scale (including writing programs & scripts, web scraping, calling APIs, write SQL queries, etc.).
Work closely with our other engineers and data scientists to integrate algorithms into our production systems.
Process unstructured data into a form suitable for analysis, automate the analysis and make the results available.
We use Spark, so experience with the ecosystem will help you hit the ground running.
Who we are looking for:
You find creative solutions to tough problems. You are an architect, who is not afraid to pave the way for bigger and better things.
Very strong programming and architectural experience, ideally in Scala, Go or Java, but we are open to other experience if you’re willing to learn the languages we use.
Automate testing, scaling, maintenance and configuration.
Strong experience building data pipelines using the Lambda architecture.
Experience with real-time processing of large amounts of structured and unstructured data.
Enough programming knowledge to clean and scrub noisy datasets.
Strong experience working with ETL using MapReduce and data streaming frameworks such as Spark and Storm is a plus as well as messaging queues.
Working with data mining, machine learning, natural language processing, or information retrieval is a plus.
Why join us?
You can work with people who are "Smart" and "Get Things Done”.
We strive to produce quality work, using the right tools and methodologies.
We embrace judicious use of automated tests.
We believe that engineers should be given freedom, as they're the most knowledgeable people to make the technical decisions.
Contribute with your own ideas and implement them.
You will have the opportunity to learn and work with a broad range of exciting technologies.
Our company consists of a highly diverse international team, and we believe that this is good.
The company is still young and growing dynamically, with a significant scope for career progression.
We offer an attractive salary and other benefits.
You can live in Berlin - culture, diversity, recreation and surprisingly low living costs.
Data Engineer (m/f) - Spark, Scala - at travel audience in Berlin, Germany at travel audience GmbH - an Amadeus Company (Berlin,
Stack Overflow · 12.09.2017