while reviewing one of the older projects that was currently implemented in the company I noticed that there was no alerting in place if the data pipeline breaks, the data pipeline has never been broken and it never failed for the last 2 years. it was a pipeline where the data would be dropped into an sftp bucket by the customer every hour and then it was moved to an s3 bucket where s3 triggers were implemented to moved the data in to queue. So in case there is an issue with the data not being processed we would only know when the we see a drop in volume. I identified it as a potential issue with the system and worked on a proposal to have monitoring in place so we are notified when there is an error. I worked with the program manager and devised a plan on how to implement the alerting and the time lines as well as the effort that will be needed to implement it. I mapped the impact of not having it to the current goals of the company and then put forward a proposal of implementing the alerts to the business and took their buyins. after getting the buyins I proposed the task to my team and some of the sr devlopers were not too keen on implenting the alerts, I gave the task to some of the Jr devlopers and the alerts were implemented. after the implementation we discovered that for some hrs in a day we werent getting any files so instead of getting 24 files in a day we were getting 20 files a day in an average. This was rectified by talking to the customer and the missing data was solved. but this made the whole team realize the importance of monitoring and alerting. It also bosted the moral of the Jr developers seeing the direct impact they made to the business goals
