Golang Software Engineer – Infrastructure & Scraping Systems Team

We are seeking a Software Engineer with a strong background in infrastructure and a passion for building robust backend systems. In this role, you will play a pivotal part in managing and evolving our core infrastructure, which is responsible for handling massive, high-load web scraping operations and integrating complex scraping systems.

Because our platform simultaneously powers operations for thousands of large-scale B2B enterprise clients, our infrastructure processes, transforms, and stores millions of data points and permanent web posts every single day. You will directly manage the high-concurrency ingestion pipelines required to scale these massive data ecosystems.

You will focus heavily on optimizing performance and cost efficiency, particularly by migrating machine learning and data extraction models from CPUs to high-performance GPU environments using Kubernetes.

Responsibilities

• Maintain, scale, and optimize the underlying core infrastructure that powers our high-volume scraping operations to effortlessly sustain the data demands of thousands of clients.
• Cooperate on transitioning machine learning models (such as logo detection) from expensive CPU setups to cost-effective GPU deployments using Kubernetes.
• Validate GPU outputs rigorously against CPU baselines to ensure scaling efficiency.
• Help identify, deprecate, and decommission legacy, inefficient scraping infrastructure to continually reduce operational overhead and streamline our high-load data pipelines.

Essential

• Strong, practical experience writing backend services in either Go or C/C++ is a strict requirement for this role.
• If your primary expertise is in C++, you must be fully willing and motivated to learn, adapt, and develop primarily in Go moving forward.
• Proven experience deploying, scaling, and managing heavy workloads using Kubernetes.
• Solid foundational experience working within AWS (Amazon Web Services) environments where our operations live.