DATA ENGINEER
Descripción de la oferta de empleo
Data EngineerJob Description.
eeking a skilled Data Engineer with a robust background in PySpark and extensive experience with AWS services, including Athena and EMR.
The ideal candidate will be responsible for designing, developing, and optimizing large-scale data processing systems, ensuring efficient and reliable data flow and transformation.
ey Responsibilities:• Data Pipeline Development.
Design, develop, and maintain scalable data pipelines using PySpark to process and transform large datasets.• AWS Integration.
Utilize AWS services, including Athena and EMR, to manage and optimize data workflows and storage solutions.• Data Management.
Implement data quality, data governance, and data security best practices to ensure the integrity and confidentiality of data.• Performance Optimization.
Optimize and troubleshoot data processing workflows for performance, reliability, and scalability.• Collaboration.
Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet business needs.• Documentation.
Create and maintain comprehensive documentation of data pipelines, ETL processes, and data architecture.
equired Skills and Qualifications:• Education.
Bachelor's or Master’s degree in Computer Science, Engineering, or a related field.• Experience.
5+ years of experience as a Data Engineer or in a similar role, with a strong emphasis on PySpark.• Technical Expertise.
Proficient in PySpark for data processing and transformation.
Extensive experience with AWS services, specifically Athena and EMR.
Strong knowledge of SQL and database technologies.
Experience with Apache Airflow is a pluso Familiarity with other AWS services such as S3, Lambda, and Redshift.• Programming.
Proficiency in Python; experience with other programming languages is a plus.• Problem-Solving.
Excellent analytical and problem-solving skills with attention to detail.• Communication.
Strong verbal and written communication skills to effectively collaborate with team members and stakeholders.• Agility.
Ability to work in a fast-paced, dynamic environment and adapt to changing priorities.
referred Qualifications:• Experience with data warehousing solutions and BI tools.• Knowledge of other big data technologies such as Hadoop, Hive, and Kafka.• Understanding of data modeling, ETL processes, and data warehousing concepts.• Experience with DevOps practices and tools for CI/CD.
Detalles de la oferta
- Axiom Software Solutions Limited
- En toda España
- Sin especificar - Sin especificar
- 07/04/2025
- 06/07/2025
Img arena is looking for sports fans in spain to collect live data from basketball games... this position is not remote, so your presence at the venue would be required to ensure accurate and timely data collection... benefits 45€ for each event covered freelancer agreement for 1 year with the possibility......
Parte de tus funciones serán: desarrollo e implementación de políticas de datos seguridad y cumplimiento datos maestros y metadatos gestión de calidad de datos evaluación y mitigación de riesgos formación y capacitación reportes innovación y mejora continua requisitos del puestoqué valoramosbuscamos......
*cvs and any other information gathered during this process will be handled confidentiallywho are we?• the center tecnològic de telecomunicaciones de catalunya (cttc) is a non-profit public sector research institution dedicated to fundamental and applied research activities, focused mainly on technologies......
Confidencial cuenta con una posición como qa engineer para garantizar la calidad del software mediante pruebas exhaustivas para identificar y corregir errores antes de su lanzamiento al mercado... identificar y reportar defectos en el software... colaborar con el equipo de desarrollo para mejorar los......
• access and management of mongodb data sources... • excellent teamwork skills... • application of scrum methodologies • use of microsoft devops for managing work items, software repositories and ci/cd• preparation of software documentation... • demonstrable experience in software development......
Requisitos del puestoroustabouts / roughnecks, welders / mechanics, rig operators / drillers, engineers (petroleum and mechanical) health and safety officer, superintendent smp, smp supervisor, document controller clerk, community superintendent, training / hrd superintendent; smp engineer, mine engineer......
Be a diploma or a degree holder... administrative and technical departmentroustabouts / roughnecks, welders / mechanics, rig operators / drillers, engineers (petroleum andmechanical) health and safety officer, superintendent smp, smp supervisor, document controller clerk,community superintendent, training......
Be a diploma or a degree holder... irequisitos del puestobe a fluent english/ french/spanish speaker... administrative and technical departmentroustabouts / roughnecks, welders / mechanics, rig operators / drillers, engineers (petroleum andmechanical) health and safety officer, superintendent smp, smp......
Business analyst, payroll manager, marketing specialist, administration supervisor, human resourcesofficer, financial analyst, senior marketing analyst, logistics coordinator / expert, procurement officer,secretary / office assistants / office clerks / front desk clerks, account / finance officer, sales......
Analizar y segmentar audiencias utilizando modelos predictivos y técnicas de data science para mejorar la eficiencia de la inversión publicitaria... dominio de herramientas publicitarias y analíticas como google ads, meta ads, tiktok ads, linkedin ads, google analytics, tag manager y data studio......
