Facebook

Hardware Systems Engineer, Storage

Facebook

September 15, 2021

Facebook is seeking a Hardware Systems Engineer to join our Release to Production (RTP) team, working on Storage platform initiatives. The RTP team is responsible for the production and sustaining of all of Facebook’s hardware in our datacenter fleet, which supports our family of applications. Our servers and datacenters are the foundation upon which our rapidly scaling infrastructure efficiently operates and upon which our innovative services are delivered. The RTP team is responsible for the end-to-end hardware lifecycle of all Facebook servers including prototyping of experimental hardware, pre-production system debugging and stress testing, enabling production system monitoring, automated provisioning and remediation of issues, and much more. RTP Engineers work closely with hardware and software co-design teams; ASIC designers, hardware designers, system manufacturers, component vendors, capacity engineering, production engineering, and many other groups at the foundation of infrastructure for the entire company. Collectively, we enable and create new systems to be deployed in our production data centers. We help to explore, develop and productize high-performance software and hardware technologies for Compute, Storage & AI in hyperscale settings. We build prototypes to demonstrate the value, enable go/no-go decisions and optimize these systems in production.
  • Interface with external vendors and internal hardware, mechanical, power, thermal, manufacturing and software engineers to understand system architecture to develop and execute the test suites for various architectures
  • Drive and own execution of full product life-cycles (prototyping, deployment, and support)
  • Proactively create experiments and tooling to detect and diagnose hardware/firmware/software health issues
  • Champion engineering and operational excellence, establishing metrics and processes for continuous monitoring, assessment and improvement
  • Troubleshoot, diagnose and identify the root cause of system failures then isolate the components/failure scenarios while working with internal & external stakeholders
  • Develop visibility through data visualization and implement systemic solutions to hardware health issues
  • Share insights with stakeholders and software teams to develop software systems to handle server failures based on hardware health data
  • Drive discussion with external and internal teams on test specification and methodologies to improve test quality
  • Bachelor in Applied Statistics, Computer Engineering, Computer Science, Electrical & Electronics Engineering, Electrical Engineering, Electronics Engineering, Mathematics and Statistics, or Mechanical Engineering
  • 5 or more years of experience in one or more domains such as: Development and analysis of storage systems (rotating media and/or solid state & archival Storage), compute (x86, ARM), AI-ML hardware (GPUs & TPUs), interconnect technologies (ex. Optics, DAC)
  • Proven solution & system level troubleshooting skills and debug experience with software, firmware and hardware problems (including issues stretching across multiple subsystems)
  • Experience with instrumenting system configurations and benchmarking change impact
  • Knowledge of server architecture and sub-components
  • Troubleshooting and analytical experience
  • Experience working in a matrix organization
  • Knowledge of Linux and x86 platforms
  • Doctor of Philosophy (Ph.D.) or Master in Applied Statistics, Computer Engineering, Computer Science, Electrical Engineering, Electrical Engineering and Computer Sciences, Mathematics and Statistics, or Mechanical Engineering
  • 5 or more years of experience related to computer and systems architecture, storage, networking or related field
  • 3 or more years of experience with Storage/compute/AI System Architecture triage and debug
  • Hyperscale experience triaging data center/infrastructure HW
  • Expertise with Python and SQL
  • Experience in solving IO bottlenecks for AI training systems
Facebook's mission is to give people the power to build community and bring the world closer together. Through our family of apps and services, we're building a different kind of company that connects billions of people around the world, gives them ways to share what matters most to them, and helps bring people closer together. Whether we're creating new products or helping a small business expand its reach, people at Facebook are builders at heart. Our global teams are constantly iterating, solving problems, and working together to empower people around the world to build community and connect in meaningful ways. Together, we can help people build stronger communities - we're just getting started. Facebook is proud to be an Equal Opportunity and Affirmative Action employer. We do not discriminate based upon race, religion, color, national origin, sex (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender, gender identity, gender expression, transgender status, sexual stereotypes, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics. We also consider qualified applicants with criminal histories, consistent with applicable federal, state and local law. Facebook is committed to providing reasonable accommodations for candidates with disabilities in our recruiting process. If you need any assistance or accommodations due to a disability, please let us know at accommodations-ext@fb.com.