Report this Job
Minimum 12+ years of experience. Be responsible for the design and support of the API(s) between IBM Spectrum Conductor (as a distributed compute and storage platform), and the Bank of America application that exposes a data scientist user experience and model governance.
Capture cluster tenant compute and storage functional and nonfunctional requirements; and translate into distributed cluster capacity, configuration, and user provisioning settings.
Develop, test, and analyze code/scripts written in PySpark, Python, Java, and other shell scripts, to provide specified behavior on a distributed IBM Spectrum Conductor cluster.
Provide "how-to" technical support for tenant developers developing runtimes and persisting data on a distributed IBM Spectrum Conductor cluster.
Be an active member of the Agile scrum, and be a part of the features that emerge from the team.
Perform peer code and test case reviews, and help foster a healthy technical community by helping peers.
Qualifications:
Experience with Agile/Scrum methodology is essential.
Experience with either Apache Spark-On-YARN (Hadoop) or Apache Spark-On-EGO (IBM Spectrum Conductor) is essential.
Experience with Apache Spark Libraries: PySpark, Spark SQL, Spark Streaming, MLlib are essential.
Experience with either Hadoop/YARN or IBM Spectrum Conductor/EGO cluster resource manager is essential.
Experience with RedHat Linux (RHEL) command line and shell scripting are essential.
Experience with file formats CSV, JSON, ORC, Avro, Parquet, Protocol Buffers are essential.
Experience with Python, Java, and R are highly desirable.
Experience with Numpy, and Pandas are highly desirable.
Experience with designing and configuring distributed architectures are desirable.
Knowledge of CI/CD SDLC practices.
Knowledge of Scikit-Learn, PyTorch, Keras, H20.ai.
Strong communication skills, should be able to communicate effectively with business and other stakeholders.
Demonstrate ownership and initiative taking
MASHPOINT LLC
