Job Description
Position: - Azure Lead Data Engineer
Location: - Warren, NJ
Job Description
Requirement:
~10+ Year experience as Data Engineer with experience in building ETL/Data pipelines with cloud technologies
~4+ years extensive experience in designing, developing, and implementing scalable data pipelines using Databricks to ingest, transform, and store structured and unstructured data.
~5+ years' experience programming languages such as Python/PySpark and query languages like SQL
~5+ years' experience in building metadata driven data ingestion pipelines using ADF
~5+ years' experience in analyzing, optimizing, and tuning existing data pipelines for performance, reliability, and efficiency
~2+ years' experience in implementing MLOps practices to streamline the deployment and management of machine learning models.
~2+ years' experience in utilizing Apache Airflow for job orchestration and workflow management
~ Familiarity with CI/CD tools and practices for automating the deployment of data engineering solutions.
~ Experience in collaborating with data scientists, analysts, and other stakeholders to understand business requirements and translate them into technical solutions
~ Knowledge/Experience in implementing security measures and standard processes to ensure data privacy and compliance with regulatory standards
~ In-depth knowledge of data engineering concepts, ETL processes, and data architecture principles.
Job Tags
Similar Jobs
NonStop Consulting NonStop Consulting
...Hybrid Work Schedule Four days onsite in Framingham, one day remote Key Responsibilities: Develop, analyze, and maintain Primavera... ..., present reports, and collaborate with project teams Review and analyze time extension requests and change management reports...
Goldman Sachs
...Asset & Wealth Management - Credit Facility Management Team Asset & Wealth Management ("AWM"): AWM invests in private equity, credit, real estate, infrastructure and other alternative investments around the world on behalf of third parties, Goldman...
Federal Maintenance Hawaii LLC
...Job Description Responsibilities: cleaning restrooms & offices Typical tasks include but are not limited to: sweeping, mopping, vacuuming, dusting, trash removal, etc. Restock restroom toilet paper, hand towels, hand soap, and seat covers, etc. Adhere to...
Sysco
...activity, and performing other related duties as required. RESPONSIBILITIES: Check, inspect, examine, and count all products being unloaded for quality and quantity against purchase orders. Inform warehouse management and merchandising of any discrepancies on...
Acupuncture Williamsburg
Acupuncturist starts out as an independent contractor making $60 per initial 1.5 hour treatment and $40 per 1 hour follow up. Patients are seen on the half hour, meaning you have the ability to see 2 patients per hour, which is $80 hourly. The right candidate will be...