I am a senior BigData developer with more than five years of experience in Java and Data development. I am used to agile software development methods and to work independently. I like to teach junior developers and leading teams by example. Because of my broad knowledge in various technologies I am often the one who develops proof of concept applications for our customers. In my spare time, I am exploring new technologies, work on GitHub projects and take pictures.

Summary

  • 5+ years of experience in development and architecture with key emphasis on the trending Big Data Technologies - Spark, Flink, Hadoop, Kafka, HBase, Hive
  • Good Knowledge of Big Data and Real-time/Offline Architecture
  • Architect, design & develop Big Data Solutions practice including set up Big Data roadmap, build supporting infrastructure and team to provide Big Data
  • Architecting and implementing Portfolio Recommendation Analytics Engine using Hive, Oozie, Spark SQL, Spark MLlib and HBase
  • Excellent understanding of Hadoop/Spark architecture and underlying framework including storage management
  • Familiar with JAVA, JVM, Scala, Zookeeper, Git, SVN, Maven, Gradle, uDeploy
  • Strong ability to tackle high performance, scalability, reliability issues
  • Comfortable with learning cutting edge technologies and applications to greenfield projects

Education

2013 - 2016

Zhejiang University

Computer Science and Technology

Professional Experience

06.2018 - 11.2019
06.2018 - 11.2019

Senior Big Data Developer (Manager)

Citi Group - CSTC (Shanghai)
  • Provided architect responsibilities including roadmaps, leadership, planning, technical innovation, security, and IT governance, etc
  • Developed technical presentations, proposals and performed presentations
  • Evaluated new technologies; Executed proof-of-concepts and developed specialized solutions
  • Designed and developed a real-time computation platform using Flink, Kafka and HBase
  • Presented and persuaded the design architecture to various stakeholders
03.2017 - 05.2018
03.2017 - 05.2018

Senior Big Data Developer (Team Lead)

Manulife (Chengdu)
  • Focused on the whole architect and development of this Recommend System project as the founder, application architect and technical leader in the Big Data team
  • Built the platform with Cloudera Manager, including Zookeeper, Hadoop, Yarn, Spark, Hive, HBase and Kafka etc.
  • Used Hive to build offline Data Warehouse, and Kafka and HBase as the real-time system
  • Used Spark MLLib to implemented recommendation system and user-loss prediction model
  • Provided technical leaderships and governance of the big data team and the implementation of the solution architecture in Big Data
  • Used Jenkins pipeline for Continuous Deployment and Jira for Continuous feedbacks
07.2016 - 03.2017
07.2016 - 03.2017

Senior BigData Developer

Yaowang Technology (Chengdu)
  • Built a user data analysis platform with a data analyst
  • Implemented architectures for all the ETL data loads coming in from the source system which were then loaded into the data warehouse
  • Built, managed and continuously monitored the Hadoop cluster through Cloudera Manager
  • Provided expertise on batch & stream with Hadoop, Kafka and Spark
  • Caught up with emerging tools and technologies and recommend
07.2014 - 06.2016
07.2014 - 06.2016

Senior Java Developer / BigData Developer

SAIC Motor Corp - Saike Electronic Commerce Co., Ltd. (Shanghai)
  • Designed and developed an online mall system
  • Designed and developed a high performance, robust and secure Big Data architecture based on Cloudera Manager ecosystems
  • Performed data cleaning and data manipulation activities using Hadoop
  • Designed and developed an online mall product recommendation system

Project History