M
Hiring companyMindrift
Active listing
Senior Python Data Scraping Engineer
Pune,Maharashtra,India Freelance Remote
About the role
Mindrift's platform connects specialists with innovative AI technology projects. As a Senior Python Data Scraping Engineer, you will create and run sophisticated web extraction pipelines, leveraging tools like Apify, Selenium, and LLM frameworks to deliver high‑quality structured data. Remote work from anywhere in India, collaborating with the Tendem project team to support real‑world AI applications.
What you’ll do
- Design and implement end-to-end data extraction workflows
- Utilize tools like Apify and custom scripts to collect and validate data
- Handle dynamic JavaScript-rendered sites and anti-bot challenges
- Scale scraping operations with batching and parallelization
- Ensure data quality through validation checks and formatting standards
What you’ll bring
- 5+ years data engineering/web scraping experience
- Proficient Python with BeautifulSoup/Selenium
- Experience with AWS and Docker
- Familiarity with LLM frameworks like LangChain or OpenRouter
- Ability to handle anti-bot mechanisms
- Strong data cleaning and validation skills
- Upper-intermediate English proficiency (B2)
- GitHub portfolio (optional)
Skills
PythonWeb ScrapingAWSDockerLLM IntegrationData ValidationSeleniumAPIs
Education
Bachelor's or Master's Degree in Engineering, Applied Mathematics, Computer Science, or related field (preferred)