Differences
This shows you the differences between two versions of the page.
| Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
| en:iot-reloaded:iot_data_analysis [2024/12/10 21:24] – pczekalski | en:iot-reloaded:iot_data_analysis [2025/05/17 08:56] (current) – agrisnik | ||
|---|---|---|---|
| Line 11: | Line 11: | ||
| === Variety === | === Variety === | ||
| - | Jain explained that big data is highly heterogeneous regarding source, kind, and nature. Having different systems, processes, sensors, and other data sources, variety is usually a distinctive feature of practical IoT systems. For instance, a system of intelligent office buildings would need data from a building management system, appliances and independent sensors, and external sources like weather stations or forecasts from appropriate external weather forecast APIs (Application programming interfaces). Additionally, | + | Jain explained that Big Data is highly heterogeneous regarding source, kind, and nature. Having different systems, processes, sensors, and other data sources, variety is usually a distinctive feature of practical IoT systems. For instance, a system of intelligent office buildings would need data from a building management system, appliances and independent sensors, and external sources like weather stations or forecasts from appropriate external weather forecast APIs (Application programming interfaces). Additionally, |
| === Veracity === | === Veracity === | ||
| Line 26: | Line 26: | ||
| ====== ====== | ====== ====== | ||
| - | Dealing with big data requires specific hardware and software infrastructure. While there is a certain number of typical solutions and a lot more customise, some of the most popular are explained here: | + | Dealing with Big Data requires specific hardware and software infrastructure. While there is a certain number of typical solutions and a lot more customised, some of the most popular are explained here: |
| === Relational DB-based systems === | === Relational DB-based systems === | ||
| Line 36: | Line 36: | ||
| * Enables asynchronous reactions to events by triggering internal events. | * Enables asynchronous reactions to events by triggering internal events. | ||
| * Data reading might be scaled out using multiple entities, while writing might be scaled up using more productive servers. | * Data reading might be scaled out using multiple entities, while writing might be scaled up using more productive servers. | ||
| - | Unfortunately, | + | Unfortunately, |
| <figure RelationalDBMS> | <figure RelationalDBMS> | ||
| Line 48: | Line 48: | ||
| Some of the most common drawbacks to be considered are: | Some of the most common drawbacks to be considered are: | ||
| * It might be scaled up only by introducing higher productivity hardware, which is limited by the application-specific design. To some extent, the design might be more flexible if microservices and containerisation are applied. | * It might be scaled up only by introducing higher productivity hardware, which is limited by the application-specific design. To some extent, the design might be more flexible if microservices and containerisation are applied. | ||
| - | * Due to the factors mentioned above and the complexity, the maintenance costs are usually higher than a universal design. | + | * Due to the factors mentioned above and the complexity, the maintenance costs are usually higher than a universal design |
| <figure CEP_systems> | <figure CEP_systems> | ||
| Line 59: | Line 59: | ||
| As the name suggests, the main characteristic is higher flexibility in data models, which overcomes the limitations of highly structured relational data models (figure {{ref> | As the name suggests, the main characteristic is higher flexibility in data models, which overcomes the limitations of highly structured relational data models (figure {{ref> | ||
| It also provides a means for scalability out and up, enabling high future tolerance and resilience. A typical approach uses a key-value or key-document approach, where a unique key indexes incoming data blocks or documents (JSON, for instance). | It also provides a means for scalability out and up, enabling high future tolerance and resilience. A typical approach uses a key-value or key-document approach, where a unique key indexes incoming data blocks or documents (JSON, for instance). | ||
| - | Some other designs might extend the SQL data models by others – object models, graph models, or the mentioned key-value models, providing highly purpose-driven and, therefore, productive designs. However, the complexity of the design raises problems of data integrity as well as the complexity of maintenance. | + | Some other designs might extend the SQL data models by others – object models, graph models, or the mentioned key-value models, providing highly purpose-driven and, therefore, productive designs. However, the complexity of the design raises problems of data integrity as well as the complexity of maintenance |
| <figure NoSQL_systems> | <figure NoSQL_systems> | ||
