Telemetry++
Make Use of Telemetry Data to Solve Problems
Across the software industry, we collect telemetry data on application and resource usage. But how many times is the collected data used in making decisions across the DevOps cycle? Data-driven decision making is still difficult to exercise in practice as data analysis is a resource and time-intensive process. That’s why we designed Telemetry++.
Telemetry++ is designed to be used by engineers and product owners to help them in surveying their applications’ usage and make inferences based on the data. It is built to have:
- A single data format for solving a diverse set of problems.
- Analysis based on hidden patterns, not only on predefined types.
- Interactive and Interpretable Visualizations.
- Quick Actionable Insights in Natural Language.
- Integration with Microsoft’s Azure App Insights Platform.
- An easy to use GUI.
Internally, Telemetry++ applies a range of advanced machine learning algorithms to solve practical problems such as:
- Finding the best deployment slots for minimizing user disruption.
- Finding Top Worflows to understand user behavior.
- Identifing Outliers/Clusters through Interpretable Visualizations.
- Finding Anomalies in Usage along with severity.
- Localizing Design/Access Problems.
What is the Input Data?
Usually, in the context of software products/services, telemetry data consists of identifiers (e.g. users) accessing different resources (e.g. Web Pages) at different instances of time (timestamps). Therefore, each row of the input data should have: an identifier of the user, a resource identifier and the timestamp. Telemetry++ processes data in CSV format. A sample of the data might look like:
| User_Id | Page_Name | Timestamp |
|---|---|---|
| 123 | Home | 2012-12-15 11:15:09.205000 |
| 233 | Auth | 2012-12-15 11:15:24.984000 |
If you are using Telemetry++ only for Time Series Analyis and Anomaly Detection, you can also pass data containing only the Timestamp and the Usage metric (e.g. Resource Usage or binned Number of Events). A sample of the data might look like:
| Timestamp | Events_Count |
|---|---|
| 2012-12-15 11:15:09.205000 | 26 |
| 2012-12-15 11:15:24.984000 | 78 |
If you are using Azure App Insights Analytics, you can provide input via the CSV file or by directly querying through Telemetry++. You can use the code snippet below to get the data.
# App Insights Analytics Code to Download the Data:
`customEvents
| where timestamp >= ago(90d)
| project user_Id, name, timestamp `
You can try the above query at App Insights Demo Page and then click the export data button, which will get you the CSV file.
To extract inights, we transform the input data into different representations throughout the application pipeline.
How do you model the problems?
More details coming soon!
Install
The application is built to work on Windows platform and is tested on Python 3 only. Currently, we are preparing it for release. It will be installable via pip.
More details coming soon!
Support or Contact
More details coming soon!
Acknowledgements and References
More details coming soon!