Technology Partners is currently seeking a talented Data Engineer (142912). Do you have experience with Data Engineering? Let us help you make your next big career move a reality!
What You Will Be Doing:
You will create and maintain optimal on-prem and cloud-based data pipeline architectures capable of ingesting structured and unstructured data using both batch & streaming methods. You will assemble large, complex data sets that meet functional and technical requirements. You will also identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc. You will build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources. You will develop data flows that can leverage both on premise and cloud architectures. You will also work with stakeholders including the business analytics teams and IS architecture teams to assist with data-related technical issues and support their data infrastructure needs.
Required Skills & Experience:
- 3+ years of experience in a Data Engineer role
- Bachelor’s degree in Computer Science, Information Systems or another applicable field is preferred
- Advanced working knowledge of cloud-based data pipeline development
- Experience building and optimizing data pipelines for acquisition and management of data including data security, metadata capture, data cataloging & classification
- Working knowledge of message queuing, stream processing and Change Data Capture mechanisms
- 3+ years of experience using the following platforms/technologies:
- Cloud-based data storage and ETL solutions
- API development (specifically to connect to SaaS solutions)
- Hadoop ecosystem of tools (Cloudera Data Hub and Flow Management)
- Relational SQL and NoSQL databases
- Change Data Capture
- Stream-processing: NiFi, Kafka, Spark-Streaming, etc.
- Security and governance: Ranger/Sentry, Navigator, Atlas
- object-oriented/object function scripting languages: Python, Java, Node.js
If this position sounds like you, WE SHOULD TALK!
We realize our people are our most valuable asset, that is why we offer the following benefits:
- Health, Dental, and Vision insurance
- 401(k) retirement plan
- Long and Short-Term disability
- Life insurance
- Direct deposit
- Referral program
Your better future is ready, and we want to put the right tools in your hands to get you there. Let’s go!
Keywords: cloud, Kafka, Hadoop, NoSQL, ETL, SQL
Looking for more opportunities with Technology Partners? Check out jobs.technologypartners.net!
Technology Partners is an Equal Opportunity Employer. Technology Partners does not discriminate on the basis of race, color, religion, sex, national origin, age, disability or any other characteristic protected by applicable state or federal civil rights laws.