Still Struggling with Tech Hiring? Discover Olibr's Solution Now!

Big Data|Remote| 4+ Years

  • Cariva Technologies
  • Remote
  • 4 - 7 Yrs
  • 15 - 22 Lacs PA

Job Description

  • Mandatory skills - Scala, SQL, PySpark.
  • Good knowledge of the business vertical with prior experience in solving different use cases in the manufacturing or similar industry
  • Ability to bring cross industry learning to benefit the use cases aimed at improving manufacturing process
  • Problem Scoping/definition Skills:
  • Experience in problem scoping, solving, quantification
  • Strong analytic skills related to working with unstructured datasets.
  • Build processes supporting data transformation, data structures, metadata, dependency and workload management.
  • Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores
  • Ability to foresee and identify all right data required to solve the problem
  • Data Wrangling Skills:
  • Strong skill in data mining, data wrangling techniques for creating the required analytical dataset
  • Experience building and optimizing ‘big data’ data pipelines, architectures and data sets
  • Adaptive mindset to improvise on the data challenges and employ techniques to drive desired outcomes
  • Programming Skills:
  • Experience with big data tools: Spark,Delta,CDC,NiFi, Kafka, etc
  • Experience with relational SQL ,NoSQL databases and query languages, including oracle , Hive, sparkQL.
  • Experience with object-oriented languages: Scala, Java, C++ etc.
  • Visualization Skills
  • Know how of any visualization tools such as PowerBI, Tableau
  • Good storytelling skills to present the data in simple and meaningful manner
  • Data Engineering Skills
  • Strong skill in data analysis techniques to generate finding and insights by means of exploratory data analysis
  • Good understanding of how to transform and connect the data of various types and form
  • Great numerical and analytical skills
  • Identify opportunities for data acquisition
  • Explore ways to enhance data quality and reliability
  • Build algorithms and prototypes
  • Reformulating existing frameworks to optimize their functioning.
  • Good understanding of optimization techniques to make the system performant for requirements.

Job Responsibilities

  • Understand the factories , manufacturing process , data availability and avenues for improvement
  • Brainstorm , together with engineering, manufacturing and quality problems that can be solved using the acquired data in the data lake platform.
  • Define what data is required to create a solution and work with connectivity engineers , users to collect the data
  • Create and maintain optimal data pipeline architecture.
  • Assemble large, complex data sets that meet functional / non-functional business requirements.
  • Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery for greater scalability
  • Work on data preparation, data deep dive , help engineering, process and quality to understand the process/ machine behavior more closely using available data
  • Deploy and monitor the solution
  • Work with data and analytics experts to strive for greater functionality in our data systems.
  • Work together with Data Architects and data modeling teams.