Azure data engineer Job at Cognizant, Pittsburgh, PA

eG1wNWlrMS93Z0kvWkJKNXByOE9vWkdqSWc9PQ==
  • Cognizant
  • Pittsburgh, PA

Job Description

Job Title - DataBricks Data Engineer

Location - Pittsburg, PA

**Must have 8+ years of real hands on experience**

We are specifically seeking a Data Engineer-Lead with strong expertise in Databricks development.

The role involves:

  • Building and testing data pipelines using Python/Scala on Databricks
  • Hands on experience to develop and lead the offshore team to perform development/testing work in Azure data bricks
  • Architect data platforms using Azure services such as Azure Data Factory (ADF), Azure Databricks (ADB), Azure SQL Database, and PySpark.
  • Collaborate with stakeholders to understand business needs and translate them into technical solutions.
  • Provide technical leadership and guidance to the data engineering team and need to perform development.
  • Familiar with Safe Agile concepts and good to have working experience in agile model.
  • Develop and maintain data pipelines for efficient data movement and transformation.
  • Onsite and offshore team communication and co-ordination.
  • Create and update the documentation to facilitate cross-training and troubleshooting
  • Hands on experience in scheduling tools like BMC control-M and setup jobs and test the schedules.
  • Understand the data models and schemas to support the development work and help in creation of tables in databricks
  • Proficiency in Azure Data Factory (ADF), Azure Databricks (ADB), SQL, NoSQL, PySpark, Power BI and other Azure data tools.
  • Implementing automated data validation frameworks such as Great Expectations or Deequ
  • Reconciling large-scale datasets
  • Ensuring data reliability across both batch and streaming processes

The ideal candidate will have hands-on experience with:

  • PySpark, Scala, Delta Lake, and Unity Catalog
  • Devops CI/CD automation
  • Cloud-native data services
  • Azure databricks/Oracle
  • BMC Control-M

Location: Pittsburgh, PA

Job Tags

Work experience placement,

Similar Jobs

Blain's Farm & Fleet

Small Engine Parts Associate Job at Blain's Farm & Fleet

 ...named a Forbes Best Employer for seven consecutive years ! Associates hired into a full-time role will become eligible for the...  ...Demonstrate awareness and compliance with Loss Prevention and safety policies and/or procedures Qualifications Great communication... 

Montavon Logistics, LLC

CLASS A OTR DRIVER NEEDED - BE YOUR OWN BOSS EARN $1600 - $2200 PR WK Job at Montavon Logistics, LLC

 ...Description Job Description Looking for teams and solo Class A drivers for Regional flatbed positions, Drivers can take the truck home...  .... The more you drive, the more you make. You can be your own boss. We don't micromanage or babysit drivers, we are looking for... 

Tri-County Service

Parts Counter Assistant Job at Tri-County Service

 ...Job Description Job Description Target Responsibilities: Sell outdoor power equipment parts to customers calling or walking into the store. Fill parts requests for the service techs which includes ordering parts. Participate in rolling inventory checks.... 

Central Oregon Truck Company

CDL A Truck Driver - $1250 orientation pay Job at Central Oregon Truck Company

 ...Hiring CDL-A Drivers YOUR ROAD TO SUCCESS - Earn up to $127,000 per year START STRONG WITH EXTRA CASH - $1,250 orientation...  ...~ Home every 2 weeks - 5 days per month ~ No Flatbed experience required for experienced OTR drivers - Get trained for Flatbed... 

Resource Informatics Group Inc

Java Backend Developer with GCP Job at Resource Informatics Group Inc

 ...Job Description Job Description Role: Backend Developers Location: Denver, CO Duration; 6+ months Rate: $market All Inclusive...  ...SQL are necessary ~ Postgres is a plus ~ Public Cloud experience a must ~ API ~ Python would be a plus...