DATA ENGINEER, HIBRIDO
Descripción de la oferta de empleo
Job Title Data Engineer
Job Description
Seeking a skilled Data Engineer with a robust background in PySpark and extensive experience with AWS services including Athena and EMR. The ideal candidate will be responsible for designing developing and optimizing large-scale data processing systems ensuring efficient and reliable data flow and transformation.
Key Responsibilities
Data Pipeline Development Design develop and maintain scalable data pipelines using PySpark to process and transform large datasets.
AWS Integration Utilize AWS services including Athena and EMR to manage and optimize data workflows and storage solutions.
Data Management Implement data quality data governance and data security best practices to ensure the integrity and confidentiality of data.
Performance Optimization Optimize and troubleshoot data processing workflows for performance reliability and scalability.
Collaboration Work closely with data scientists analysts and other stakeholders to understand data requirements and deliver solutions that meet business needs.
Documentation Create and maintain comprehensive documentation of data pipelines ETL processes and data architecture.
Required Skills and Qualifications
Education Bachelors or Master s degree in Computer Science Engineering or a related field.
Experience 5+ years of experience as a Data Engineer or in a similar role with a strong emphasis on PySpark.
Technical Expertise
o Proficient in PySpark for data processing and transformation.
o Extensive experience with AWS services specifically Athena and EMR.
o Strong knowledge of SQL and database technologies.
o Experience with Apache Airflow is a plus
o Familiarity with other AWS services such as S3 Lambda and Redshift.
Programming Proficiency in Python experience with other programming languages is a plus.
Problem-Solving Excellent analytical and problem-solving skills with attention to detail.
Communication Strong verbal and written communication skills to effectively collaborate with team members and stakeholders.
Agility Ability to work in a fast-paced dynamic environment and adapt to changing priorities.
Preferred Qualifications
Experience with data warehousing solutions and BI tools.
Knowledge of other big data technologies such as Hadoop Hive and Kafka.
Understanding of data modeling ETL processes and data warehousing concepts.
Experience with DevOps practices and tools for CI/CD.
ETL, CI/CD, DevOps, Kafka, Hive, Hadoop, AWS, Lamda, Apache, SQL, EMR, PySpark
ETL, CI/CD, DevOps, Kafka, Hive, Hadoop, AWS, Lamda, Apache, SQL, EMR, PySpark
Detalles de la oferta
- Axiom Software Solutions
- Sin especificar
- 06/01/2025
- 06/04/2025
Our division focuses on gathering, enriching and processing data for machine learning in different ai domains... we offer high-quality data for human-machine interaction to some of the most prestigious technology companies in the world... position: data contributor project location: barcelona, spain......
Parte de tus funciones serán: desarrollo e implementación de políticas de datos seguridad y cumplimiento datos maestros y metadatos gestión de calidad de datos evaluación y mitigación de riesgos formación y capacitación reportes innovación y mejora continua requisitos del puestoqué valoramosbuscamos......
Img arena is looking for sports fans in spain to collect live data from basketball games... this position is not remote, so your presence at the venue would be required to ensure accurate and timely data collection... benefits 45€ for each event covered freelancer agreement for 1 year with the possibility......
€ eur por año bruto jornada laboral: de lunes a viernes modalidad: hibrido horario: 9am a 6pm tipo de contrato: contrato indefinido prestaciones: prestaciones de ley... confidencial cuenta con una posición como qa engineer para garantizar la calidad del software mediante pruebas exhaustivas para......
*cvs and any other information gathered during this process will be handled confidentiallywho are we?• the center tecnològic de telecomunicaciones de catalunya (cttc) is a non-profit public sector research institution dedicated to fundamental and applied research activities, focused mainly on technologies......
Com/project/tahoe-nl?/fill?id313=df035 join our data collection project tahoe in barcelona, spain! project overview: purpose: collect voice recordings to improve speech recognition technologies... otros datos del puestoreferral program join our referral program! earn 25 usd for each referred participant......
¡no lo dudes y únete a nuestro equipo!requisitos del puesto... valorable nivel medio de inglés (b2)... jornada completa... 000 euros brutos anuales (+ bonos de productividad)... posibilidad de crecer en una empresa de alcance internacional... flexibilidad horaria en turnos rotativos... com/language/es/empleo/envianos-tu-curriculum/]......
¡no lo dudes y únete a nuestro equipo!requisitos del puesto... valorable nivel medio de inglés (b2)... jornada completa... nivel alto/bilingüe de alemán (c1/c2)... 000 euros brutos anuales (+ bonos de productividad)... posibilidad de crecer en una empresa de alcance internacional... com/language/es/empleo/envianos-tu-curriculum/]......
Otros datos del puesto보상: 과제 a: 최대 2시간 녹음 세션에 대한 75달러 과제 b: 최대 1시간 녹음 세션에 대해 55달러 사용 가능한 결제 방법: (선호하는 결제 수단에 따라 paypal, 기프트 카드 또는 전신 송금을 선택할 수 있습니다)... 제목: 데이터 수집 프로젝트 타호 직위: 데이터 제공자 프로젝트 위치: 스페인 바르셀로나 (현장, 예약된 약속에 따라) 작업 기간: 최대 2시간 참여 모델: 일회성 작업, 작업 기반 신청 버튼: https://dataforcecommunity... com/project/tahoe-es?/fill?id313=df024......
Food and restaurant department bakers / bartenders / chefs / chef de partie / sous chef / chef d 'cuisine / cooks / dietitians, food and beverage servers, food service supervisors, restaurant and food service manager, catering manager, assistant beverage manager, kitchen manager, banquet coordinator......
