Data Engineer (Full-Remote in Japan*) at PayPay Corporation. (Tokyo, Japan) (allows remote)

Data Engineer (Full-Remote in Japan*) at PayPay Corporation. (Tokyo, Japan) (allows remote)


Add To Bookmarks
Company:
Location: Tokyo, Japan
Type: Full Time
Created: 2021-05-08 05:00:23

Apply Here


PayPay is looking for Data Engineer to work on our payment system to deliver the best payment experience for our customers.

  • Work directly with Data Analysts and Platform Engineering Team to create reusable experimental and production data pipelines
  • Understand, tune, and master the processing engines (like Spark, Hive, Cascading, etc) used day-to-day
  • Keep the data whole, safe, and flowing with expertise on high volume data ingest and streaming platforms (like Spark Streaming, Kafka, etc)
  • Sheppard and shape the data by developing efficient structures and schema for the data in storage and transit
  • Explore as many new technology options for data processing, storage, and share them with the team
  • Develop tools and contribute to open source wherever possible

Tech Stack We select the best combination of tech at times. | Scala, Java, Python| PlayFramework| Aurora, DynamoDB, TiDB, Spark, Presto, Hive, Hudi, Airflow, Kafka| Docker, Kubernetes,AWS, GCP| GitHub,IntelliJ| Presto, Athena, Bigquery, Looker,Spark, Hive| Draw.io| Slack, Zoom, F2F

Qualifications

  • Experience with building data pipelines ingesting and transforming large number of events per minute and terabytes of data per day.
  • Experience with Spark and Kafka and have experimented or heard about Flink/Druid/Ignite/Presto/Athena and understand when to use one over the other. (Preferably, experience of working with cloud based big data processing platform such as AWS EMR, Google Cloud DataProc)
  • Passionate about producing clean, maintainable and testable code part of real-time data pipeline.
  • Good understanding of microservices work.
  • Ability to connect different services and processes together even if you have not worked with them before and follow the flow of data through various pipelines to debug data issues.
  • Good understanding of issue with ingesting data from applications in multiple data centers across geographies, on-premise and cloud and will find a way to solve them.
  • Proficient in Java/Scala/Python/Spark

Preferred Qualifications

  • Software development experience in one or more general purpose programming languages, preferably Scala.
  • Experience with implementing platform components such as RESTful APIs, Pub/Sub Systems, Database Clients.
  • Experience with AWS services.
  • Contribution to open source projects.
  • Knowledge about the Japanese payment industry

Hiring Process

  1. Application Review (1-2 weeks)
    • HR and Team will review your resume
  2. Code challenge (online)
    • Coding Test will be sent via Hirevue system
    • It takes 3-4 hours (max) to complete
    • If you need to extend the due date, please contact HR
  3. Interviews (online)
    • 2-3 rounds of online interview(s)
    • Live Coding could be requested
    • Please make sure the reason you applied to PayPay (Why Fintech? Why Startup? Why PayPay?)
  4. Job Offer

*Relocation to Japan

  • Due to the current COVID-19 situation, we cannot sponsor working VISA to Japan. However as a temporary solution, you may be able to start working with us as an individual contractor. Please discuss with your recruiter about this opportunity.
  • Once the COVID-19's over, we will ask all employees in overseas to relocate to Japan. We will fully support your relocation.

Other Information

[Corporate Blog] https://about.paypay.ne.jp/corporate-blog

[Product Blog] https://blog.paypay.ne.jp

[LinkedIn] https://www.linkedin.com/company/paypay-corp/