Automation Engineer - Worlwide Remote
Invisible TechnologiesUnited States only
Apply now
Opens the original posting on Himalayas.
Apply nowSource: HimalayasApply on the original site
Companies in this catalog do not charge applicants. Never send money to apply.
Job Responsibilities:
- Create crawlers for each source URL by using Python modules [scrapy, selenium, requests, BeautifulSoup and splash]
- Create and maintain scrapy pipelines and middlewares to manage the output from the crawlers.
- Create crawlers for all types of websites irrespective of the technical roadblocks.
- Manage the crawlers to overcome technical challenges like IP ban, geolocation ban, captcha and bot blocking services.
- Write SQL queries to manage database operations using Python modules like sqlalchemyDeploy the python scripts / crawlers to Linux based AWS servers