My Practicum project for SGC was my work experience with the Data Engineering team at A.M. Best Rating services. I was offered an internship during summer 2023 and gladly accepted. My manager during the internship was Rohit Motiani and I was working with the rest of the data engineering team, assisting and learning where I could. I was also one of the 10 interns in the summer internship program, and we did team-building activities together, like a mock interview at the recording studio, buying ice cream, and doing park cleanup activities.
As for the technical work, it was a very steep learning curve. I was told that my job was to learn about Apache Airflow, which is a data orchestration software. I unfortunately had to spend almost half of the internship just getting it installed and running. I had many problems, such as requiring administrative privileges on my work laptop in order to install WSL (as Apache Airflow was a Linux-only software, and the work laptop was running Windows), dealing with networking problems due to the VPN we had to use for security reasons, and general performance issues due to the fact that the laptop was not powerful enough for a smooth running Airflow instance. Oh the side, though, I was tasked with learning SQL and databases. I did this, and it helped me tremendously in the long run. In the process of learning to install Airflow, I learned WSL (which I use to this day), better Linux use, and Docker. I then moved on to learning how to use Airflow and Pandas. My knowledge of widely used software increased tenfold in the span of a few weeks.
I then did a demo showing what I learned and how it could be applied to our needs. I was then tasked with writing an ETL (Extract, Transform, and Load) data pipeline which could automatically move data from the database to the cloud. This project was a culmination of everything I learned, and I was severely behind schedule due to the delays, so I ended up having it working in a few days. It involved a lot of logic, thinking it through, and debugging, which I improved from due to it being a real project. I gave a demo on this as well, and it went great. My presentation skills also improved from the demos I did. I also did my final demo with the other interns in front of a lot of people like managers, HR, and some executives.
After the internship (and also towards the end of it), my manager spoke to me about potentially continuing my work. After he talked to HR, I was told it was not possible, but I might be able to work later as a part-time contractor during the fall 2023 semester, which I ended up doing. This time, my manager was Matthew Coppola, but I worked in the same team so not that much changed. During this part time work, I gained experience with Microsoft Azure Cloud services, and Cloud skills are important to know if you are in the tech industry today, so it's yet another important thing I learned.
I think I learned quite a lot from these experiences. I saw the high levels of what data workflows looked like and why businesses need IT teams. I also think that I learned more about my personal interests and how I want to advance my career. Data engineering was very interesting, but I'm not sure if it is something I can see myself doing for a long time. I am more interested in the data science or analysis side of things, but I have recognized that without data engineering, none of that would be possible at scale. However, A.M. Best will remain a very good potential choice in my career after graduation.
I think that good advice for future SGC students going into the STEM field is to not always focus on internships and jobs at extremely large companies, and to additionally try for small companies that are near where you live (it was around a 50 minute drive from my home). A.M. Best also apparently provides a very stable career according to the people that work there, with some people staying at the company for 30+ years. Smaller companies are absolutely a valid choice and often have many advantages.