Data Engineer
Via agency · IDBC Group
Software developer
Database administratorSystems analystBudapest
IDBC Group is looking for an experienced Data Engineer to work in Budapest. Responsibilities include designing and developing data architectures and data pipelines, as well as creating data products for analysts and data scientists.
Tasks
- Design, develop, optimize, and maintain data architectures and data pipelines in accordance with ETL and data lake principles.
- Prepare, coordinate, and hand over artifacts related to data architecture and data pipelines to the platform team so they can be reused by other teams.
- Solve complex technical data management challenges to support business goals.
- Develop data products for analysts, data scientists, and machine learning engineers that increase team and organizational efficiency.
- Support, mentor, and provide professional guidance to data and analytics professionals in applying data management standards and best practices at the team and organizational level.
- Contribute to the evaluation of new tools and technologies in data engineering and data science, and help define and improve working methods and standards.
- Promote continuous improvement through training, competency development programs, and the enhancement of analytical data engineering practices, standards, and processes.
Requirements
- Degree in IT or a related field.
- 4-10 years of experience in data engineering, within distributed computing, big data, or advanced analytics environments.
- High level of SQL and data analysis skills, plus confident use of at least one programming language (e.g., Python or Scala).
- Experience in database development and data modeling, ideally in Databricks/Spark and SQL Server environments; knowledge of relational, NoSQL, and cloud databases.
- Confident knowledge of distributed computing systems, with experience using Spark or MapReduce technologies as an advantage.
- Hands-on experience with Azure-based services such as Azure Data Factory, Azure Databricks, Event Hub, Synapse, and Machine Learning services.
- Good general knowledge of core data and analytics concepts, including dimensional data modeling, ETL processes, data warehouses, reporting, data governance, and the management of structured and unstructured data.
- Knowledge of Unix systems, with a particular focus on shell scripting.
- Basic networking knowledge, connection and troubleshooting experience.
- Basic knowledge in machine learning, data science, artificial intelligence, statistics, or applied mathematics.
- Excellent communication skills and confident English proficiency; knowledge of German is an advantage.