As the use of AI becomes more widespread across various industries, the need to monitor AI-driven applications for anomalies and unexpected behaviors has become increasingly important. Each use case is different and may require a unique set of fields and metrics to effectively identify and surface anomalous behaviors early before business is negatively impacted. At Mona, we are committed to providing the most advanced AI monitoring platform to improve the accuracy and reliability of AI-based systems. We have developed a one-of-a-kind insight generator designed to detect the specific data segment in which the anomaly lies, providing users with the full context of the behavior including possible explanations on why it is occurring. The key to successful AI monitoring lies in the ability to adhere to the intricacies of each use case, providing users with valuable insights to optimize their models and processes. Following numerous user requests, we just enhanced our monitoring capabilities to use geo-location and multiple timestamp fields in any monitoring context.
Monitoring Multiple Time Dimensions with Mona
From day one, Mona’s intelligent monitoring platform has always supported time series information, a crucial aspect for detecting both gradual drifts and sudden changes in data. As our monitoring platform continued to serve more users, we recognized the need for multiple time dimensions in many AI-based processes.
As an example, let’s explore a common fintech use-case for AI - approving loan applications. When monitoring such an AI-based process, it’s crucial to track several specific times across the process, which can be anywhere between seconds to even weeks apart. These times may include the time the loan application was made, the times the credit and fraud models started running on the application, and the time that the first monthly payment was paid back.
If a user returns their first payment, there is a process that aids in calculating their credit score for future loans. However, if a bug occurs during this process, it may not be immediately apparent when the bug occurred unless you track that entire process according to the time it took place (e.g., if it took one month after the original loan application). Without monitoring multiple time dimensions, it will be difficult to pinpoint the exact time when the bug occurred. Mona is able to support multiple time dimensions, enabling users to monitor all of these timestamps in one place.
With Mona, you can segment your data according to different weekdays, run different drift detections on different time fields, and define duration fields, tracking the distances between time fields. By utilizing these features, you can quickly identify potential anomalies and make timely decisions to optimize your AI-based processes.
Geo-Based Monitoring Capabilities for AI-Based Processes
Geo information is often overlooked, but it can be crucial for many processes. Mona now supports adding exact geo locations and polygonal geo shapes to provide automatic geo grid segmentation to find specific locations where anomalies occur.
One example of requiring such capabilities is for monitoring AI that’s running on aerial imagery. In fact, this was the case with a Mona user that had their image recognition models underperforming in some cases but couldn’t find common threads for the misbehavior. By adding the location of the images to the tracked information, Mona was able to automatically find the specific locations where the underperformance occurs, and make it easy to understand that the models are not as well trained on images taken in mountainous terrains.
Similar examples were found for users doing AI for healthcare (finding specific areas were automatic diagnosis of Covid didn’t work as well), fintech (finding undetected frauds when using IP addresses originating from specific locations) and ecommerce (churn detection underperformance for users coming from rural areas).
In all these cases and more, having a geographical understanding of the data helped in resolving the issues more quickly and improving research capabilities.
Conclusion
Mona’s intelligent monitoring capabilities now includes geo-location and multiple timestamp fields to provide users with comprehensive insights into their data, enabling users to optimize the performance of their models. As Mona continues to grow and encounter other data requirements, we will develop more advanced capabilities to make it easier to monitor AI processes. Request a demo today to improve the reliability of your ML models!
