Telemetry Data

Intro to Telemetry Data

Telemetry is the remote collection and transmission of information in the form of measured values. Telemetry data is collected from a variety of sources, including remote or hard-to-reach places (e.g., space, inside equipment). In addition, it can come from internal systems (e.g., data generated from computers or IoT devices) or purpose-built sensors embedded in systems or standalone.

Telemetry data is commonly transmitted through wireless mechanisms, such as radio, ultrasonic, or infrared systems, but it can also be transferred over other media, such as optical links, telephones, computer networks, and other wired systems. The data can be streamed or collected at the point of origin and sent in batches to other systems to be monitored, displayed, analyzed, recorded, and stored.

Telemetry Data and the Cloud

Cloud telemetry uses software to gather, record, and analyze information about systems that are difficult to access due to location and scale. With cloud telemetry data, IT teams can remotely detect and respond to issues as well as analyze systems to track and optimize performance, utilization, and energy consumption. Insights from telemetry data can also help reduce costs, enhance security, and improve the overall efficiency of operations.

Logs, metrics, and traces make up the bulk of cloud telemetry data.

• Logs
  • Mostly unstructured data
  • Event-based records of any notable anomalies across the system
  • Results of any transaction involving an endpoint within a cloud or multi-cloud environment
• Metrics
  • Data points represented as counts or measures that are often calculated or aggregated over a period of time
  • Originated from several sources, including infrastructure, hosts, and third-party sources
  • Accessible via query
• Traces or following a process (e.g., an API request or other system activity)
  • Show services connect
  • Provides insights into how the cloud ecosystem is working
  • Includes span data, such as unique identifiers, operation names, timestamps, logs, events, and indexes

Other cloud telemetry data includes:

• Business logic data—e.g., number of sales transactions, revenue, user signups, churn rate, A/B testing results from marketing campaigns
• Application layer data—e.g., transaction times, latencies, response codes, errors
• Infrastructure layer data—e.g., CPU load, memory consumption, disk space, network bandwidth
• User software-level data—e.g., application errors, crashes, performance

An Open-Source Approach to Cloud Telemetry

Modern cloud-native applications are distributed, making the capture and export of telemetry data complicated. OpenTelemetry’s single set of standards and technology tools make it easier to capture data and export it to participating technology vendors or open-source projects.

OpenTelemetry,  A Cloud Native Computing Foundation incubating project

Telemetry Data Types

The breadth of data sources ranges from inside people or animals to inside almost any kind of machine, making the benefits of telemetry data nearly limitless. Below are some of the near-infinite sectors that generate telemetry data and examples of the sources.

• Agriculture
  Report on conditions (e.g., soil moisture) and control sprinklers and other systems
• Automobiles
  Remotely monitor fuel level, engine heat, and vehicle speed
• Energy
  Manage electrical grids and power generation facilities (e.g., dams, wind farms)
• Geology
  Measure seismic activity
• Infrastructure
  Monitor pipelines and water systems 
• Medicine
  Monitor patients using various medical devices  
• Mobile apps
  Monitor patients’ vital signs from a smartwatch
• Research
  Tag animals with locators to track their migration patterns and locate them for study
• Retail
  Track retail inventory
• Space
  Relay data from outer space (e.g., spacecraft, rovers, satellites)
• Technology
  • Computers
  • IoT
  • Robots
    

How Telemetry Works

Telemetry data is collected from sensors at remote sources. A typical telemetry system consists of an input device called a transducer, a medium of transmission (usually radio waves), equipment for receiving and processing the signal, and storage systems.

The Transducer

• Converts the physical stimulus to be measured (e.g., temperature, vibration, pressure) into an electrical signal
• Operates as the measuring instrument
• Can be self-generating or externally energized
• Transmits in two ways:

1. 1. Radio communication is used when it is impractical to provide wire lines.
2. 2. Cable and line links are used in built-up areas where it is difficult to find antenna sites and unobstructed line-of-sight radio paths.

Transmission

• Used to impress the information on the carrier frequency and the subcarrier
• Has a direct influence on system performance
• Tends to define areas of design in both the sender and the receiver
• Has modulation methods that fall into two divisions:

1. 1. Amplitude
2. 2. Frequency modulation (i.e., commercial AM and FM broadcasting) and related types

Processing the Received Signal

Telemetry data flows out of devices primarily as unstructured data. This information needs to be:

• Collected
• Put into an appropriate structure for storage
• Combined with other data (at times)
• Stored in a transactional database
• Transferred to an analytics-oriented database or analyzed in place

Benefits of Telemetry

The primary benefit of telemetry data is that it provides access to embedded raw information that, when extracted, can be analyzed and turned into actionable insights. It can also be sent from remote locations. Following are some of the many benefits of telemetry data that can be gained by pulling data from one or multiple sources. 

• Collect data from moving objects
  Using telemetry data collected from cameras, trends can be identified by analyzing the behavior of vehicles, animals, or people to aid in a range of areas from research to law enforcement.
• Gain early insight into natural disasters 
  Telemetry data from sensors and weather balloons can provide advance notice of physical changes that could be a sign of an impending natural disaster, such as tornadoes or tsunamis.
• Improve technical operations
  Remote automated systems monitoring servers and other equipment’s telemetry data can trigger alerts that allow for preemptive issue resolution and lower mean-time-to-recovery (MTTR)—from power plants and bridges to factories and data centers. 
• Monitor health status
  Whether at a medical facility or as an outpatient, telemetry data from people can be remotely monitored by healthcare workers, allowing for expedient responses to any immediate or potential problems. 
• Reduce operational costs
  Because telemetry data can be broadcast, the expense of manually collecting information from multiple devices is reduced or eliminated. 
• Share data
  Raw telemetry data can easily be shared across systems and teams to develop rich data sets and support advanced analytics.

By making effective use of telemetry, organizations can improve key performance indicators, including TCO, reliability, security, performance, and power consumption.

Intel

Challenges of Telemetry

• Create, manage, and maintain systems that can analyze the large volume of telemetry data (e.g., IoT data generated in a year is measured in zettabytes, a unit equal to one trillion gigabytes)
• Face data governance and data quality issues due to the volume of data
• Fend off increasing threats of data breaches and malware attacks that come with the growing scale and volume of data generation
• Implement management and protection systems for sensitive data
• Protect the privacy rights of citizens (e.g., sensitive data collected from systems and cameras)

Telemetry—The Whole Is Greater than the Sum of Its Parts

Coming in small packets of 512 bytes, the value of telemetry data multiplies with its volume—arguably exponentially. Telemetry data has been collected and analyzed since the early 20^th century, when information about the operation of electric power plants was transmitted over telephone lines.

As the capability and capacity to collect and analyze telemetry data increases, so do the insights it provides. Telemetry data is collected and used across all sectors, with information coming from every source imaginable. The power of telemetry data continues to increase as machine learning and artificial intelligence (AI) are brought to bear on growing telemetry data stores.

Egnyte has experts ready to answer your questions. For more than a decade, Egnyte has helped more than 16,000 customers with millions of customers worldwide.

Last Updated: 23rd December, 2021

Get started with Egnyte.

Request Demo