Job Description
Job description
Scope of Position:
The position is to provide both web scraping application support and new development of web scrapers and other automation methods for our in-house systems. Also, the candidate will become a product and service Subject Matter Expert (SME) for our Automation applications. This role will monitor, support, maintain and build web scrapers within the systems and will require this individual to manage, map, resolve, and consult on data coming to and leaving the Automation applications. This role will strive to optimize processes and data flow for efficient operations and standards. The job provides great opportunity for candidate developing their career path in various technical/technologies platforms as a part of a global team and coordinate with team located in US, HK and Malaysia.
Essential Job Duties:
- Analyze and steward data to insure that data will comply with user and application requirements and maintain data integrity standards.
- Understand data from all products.
- Support and maintain web scrapers primarily in Python for the Automated Investigations Management System (AIMS).
- Support and maintain the in house built Scrapy and Callidus Systems
- Develop scrapers with Scrapy and Callidus that will crawl the web and streamline business processes
- Analyze data to insure that extracted web content will comply with user requirements
- Monitor the performance and stability of services
- Work closely with end users to define needs/requirements of web scrapers
- Ensure effective communication between all necessary individuals.
- Document accurately and ensure material is available for company use.
-
Education & Qualification Requirements:
- UG : B.Tech/B.E. - Computers, B.Sc - Any Specialization, Computers
- PG : MBA/PGDM - Any Specialization, Information Technology, M.Tech - Any Specialization, Computers, MCA - Computers
- Experience: 5 to 10 years, Office working hours: 2:00 PM to 11:00 PM
Required Skills:
- 3+ years of experience in Python
- 2+ years of experience in JavaScript
- Proficient in HTML, XML, XPath, XSLT and Regular Expressions at a data level
- Experience with MS SQL Server 2005/2008/2012 databases. Good SQL skills a major plus
- Experience with web scraping and/or Scrapy Framework a major plus
- Experience with .NET, Puppeteer, Selenium a plus
- Experience with Linux a plus
- Highly motivated, assertive self-starter that is goal-driven and detail-oriented
- Team-oriented and dependable with well-developed interpersonal skills and work habits.
- Strong interpersonal and communication skills
- Capable of working extended work hours based on systems or client needs
- Ability to work overtime when necessary
- Training will be provided for AIMS and writing in Scrapy and Callidus
Expectations:
- Job Knowledge Competent in required job skills and knowledge. Exhibits ability to learn and apply new skills. Keeps abreast of current developments. Requires minimal supervision. Displays understanding of how job relates to others. Uses resources effectively.
- Quality Strong attention-to-detail. Must have strong time management skills, the ability to multi-task and work at a fast pace. Demonstrates accuracy and thoroughness. Displays commitment to excellence. Looks for ways to improve and promote quality. Applies feedback to improve performance. Monitors own work to ensure quality.
- Problem Solving Identifies problems in a timely manner. Gathers and analyzes information skillfully. Develops alternative solutions. Resolves problems in early stages. Works well in group problem solving situations.
- Analytical Skills Synthesizes complex or diverse information. Collects and researches data. Uses intuition and experience to complement data.
- Teamwork Balances team and individual responsibilities. Exhibits objectivity and openness to others views. Gives and welcomes feedback. Contributes to building a positive team spirit. Puts success of team above own interests.
- Communication –Excellent written and verbal communication skills. Edits work for spelling and grammar. Speaks clearly and persuasively. Listens and gets clarification. Participates in meetings.