Alternative Path

Job Description:

Leading and mentoring a team of seasoned Data engineers performing Web Scraping using various scraping techniques and then utilizing Python's Pandas library for data cleaning and manipulation. Then ingesting the data into a Database/Warehouse, and scheduling the scrapers using Airflow or other tools

 Technical Leadership

1. Oversee the design and implementation of web scraping projects to ensure scalability, efficiency, and accuracy.

2. Stay updated with and implement the latest technologies, tools, and frameworks in web scraping and data processing.

3. Review and approve pull requests to ensure clean, maintainable, and efficient code.

4. Identify and solve complex technical challenges in data extraction, handling, and storage.

5. Design and implement monitoring tools and dashboards to ensure system reliability and performance.

6. Experience in the Data Engineering or Web Scraping industry, with a strong background in data processing and automation.

7. Experience in leading teams, mentoring engineers, and driving technical initiatives

 Team Management (Techno-Managerial Role)

1. Lead, mentor, and inspire a team of engineers to achieve project goals and professional growth.

2. Conduct regular one-on-ones to provide feedback, set objectives, and discuss career development.

3. Monitor team productivity and allocate resources effectively to meet deadlines.

4. Foster a collaborative and high-performing team culture that promotes innovation and ownership.

5. Serve in a techno-managerial role, balancing technical expertise with leadership responsibilities to drive both project execution and team development.

6. Collaborate with stakeholders to understand project requirements and translate them into actionable plans.

7. Develop long-term strategies for web scraping solutions, including data storage, compliance, and scalability.

8. Drive continuous improvements in process, tools, and methodologies for the team.

9. Align team efforts with business goals and identify opportunities for automation and process optimization.

10. Ensure high-quality deliverables by enforcing best practices in coding, testing, and documentation.

11. Optimize the data pipeline to handle large-scale data ingestion and cleaning.

12. Implement safeguards for compliance with web scraping regulations and ethical practices.

13. Ensure the team adopts the best practices for handling sensitive and proprietary data.

Strategic Planning and Execution

1. Collaborate with stakeholders to understand project requirements and translate them into actionable plans.

2. Develop long-term strategies for web scraping solutions, including data storage, compliance, and scalability.

3. Drive continuous improvements in process, tools, and methodologies for the team.

4. Align team efforts with business goals and identify opportunities for automation and process optimization.

 Operational Excellence

1. Ensure high-quality deliverables by enforcing best practices in coding, testing, and documentation.

2. Optimize the data pipeline to handle large-scale data ingestion and cleaning.

3. Implement safeguards for compliance with web scraping regulations and ethical practices.

4. Ensure the team adopts the best practices for handling sensitive and proprietary data.

Visa

Job Description
 
The Opportunity: 
As a Senior Manager, you will be part of Risk and Identity Solutions team and will provide hands-on technical leadership to the team to build our applications in an agile development environment by hiring and building a world-class team of engineers, defining, executing and delivering functional and non-functional features at scale quickly and promoting a culture of cross-functional collaboration and engineering excellence.

 
The Work itself: 
•    Provide hands on technical leadership and oversight to a team of software engineers (Full time employees and contractors)
•    Be part of the development team with hands on coding skills and excellent collaboration skills.
•    Deliver the product with moderately complex technical interdependencies with cross geographical teams.
•    Actively participate in Solution design, architecture and development of key initiatives. Ensure developed solutions adhere to established architectural best patterns and target state. 
•    Foster the culture of Engineering Excellence (quality, security, performance, scalability, availability, resilience etc.), CI/CD, Automation and Shift-Left.
•    Hire, retain and grow a high-performing team of software engineers.
•    Invest in the career development of direct reports.
•    Create an engineering culture of creativity and innovation.
•    Provide technical leadership to development team by participating in design, performing design reviews and code reviews.
•    Provide technical leadership for quality assurance and test engineering activities – manual and automation
•    Lead scrum ceremonies and program management activities. Play a crucial role in interfacing with business stakeholders to define solutions and requirements, prioritize, and drive execution using agile methodologies.
 

Qualifications

Basic Qualifications
• 12+ years of relevant work experience with a Bachelor’s Degree or with an Advanced Degree 

Preferred Qualifications
• Experience in managing technical deliveries required.
• Expertise with the Building and managing Data pipeline application and architecture
• Expertise with Hadoop, Hive, Kafka, Spark & Java
• Expertise maintaining & optimizing data pipelines
• Proven experience delivering highly scalable and available production systems.
• Understanding of architecture and operations of highly available and scalable transactional systems.
• Ability to manage multiple competing top priorities in a fast-paced environment.
• Experience working with diverse teams and stakeholders
• Ability to manage multiple competing top priorities in a fast-paced environment.