Differences

This shows you the differences between two versions of the page.

Link to this comparison view

--- en:safeav:avt:challenges [2025/08/01 23:49] – rahulrazdan
+++ en:safeav:avt:challenges [2025/10/20 18:01] (current) – raivo.sell
@@ Line 2: / Line 2: @@
 {{:en:iot-open:czapka_m.png?50| Masters (2nd level) classification icon }}
-<todo @raivo.sell></todo>
+<todo @raivo.sell #raivo.sell:2025-09-18></todo>
+{{:en:safeav:avt:av_stack.jpg?600|}}
+In terms of challenges, autonomy is very much in the early innings. Broadly speaking. the challenges can be split into three broad categories. First, the core technology elements within the autonomy pipeline (sensors, location services, perception, and path planning, the algorithms and methodology for demonstrating safety, and finally business economics.
+Autonomous vehicles rely on a suite of sensors—such as LiDAR, radar, cameras, GPS, and ultrasonic devices—to perceive and interpret their surroundings. However, each of these sensor types faces inherent limitations, particularly in challenging environmental conditions. Cameras struggle with low light, glare, and weather interference like rain or fog, while LiDAR can suffer from backscatter in fog or snow. Radar, though more resilient in poor weather, provides lower spatial resolution, making it less effective for detailed object classification. These environmental vulnerabilities reduce the reliability of perception systems, especially in safety-critical scenarios. Another major challenge lies in the integration of multiple sensor types through sensor fusion. Achieving accurate, real-time fusion demands precise temporal synchronization and spatial calibration, which can drift over time due to mechanical or thermal stresses. Furthermore, sensors are increasingly exposed to cybersecurity threats. GPS and LiDAR spoofing, or adversarial attacks on camera-based recognition systems, can introduce false data or mislead decision-making algorithms, necessitating robust countermeasures at both the hardware and software levels. Sensor systems also face difficulties with occlusion and semantic interpretation. Many sensors require line-of-sight to function properly, so their performance degrades in urban settings with visual obstructions like parked vehicles or construction. Even when objects are detected, understanding their intent—such as whether a pedestrian is about to cross the street—remains a challenge for machine learning models. Meanwhile, high-resolution sensors generate vast data streams, straining onboard processing and communication bandwidth, and creating trade-offs between resolution, latency, and energy efficiency. Lastly, practical concerns such as cost, size, and durability hinder mass adoption. LiDAR units, while highly effective, are often expensive and mechanically complex. Cameras and radar must also be ruggedized to withstand weather and vibration without degrading in performance. Compounding these issues is the lack of standardized validation methods to assess sensor reliability under varied real-world conditions, making it difficult for developers and regulators to establish trust and ensure safety across diverse operational domains.
+{{:en:safeav:avt:sensors-21-05397-g020-550.jpg?600|}}
+The "perception system" is at the core of autonomous vehicle functionality, enabling the car to understand and interpret its surroundings in real time. It processes data from multiple sensors—cameras, LiDAR, radar, and ultrasonic devices—to detect, classify, and track objects. The perception system struggles with "semantic understanding and edge cases." While object detection and classification have improved with deep learning, these models often fail in rare or unusual scenarios—like an overturned vehicle, a pedestrian in costume, or construction detours. Understanding the context and intent behind actions (e.g., whether a pedestrian is about to cross) is even harder. This lack of true situational awareness can lead to poor decision-making and is a key challenge for Level 4 and 5 autonomy. Also, the "computational burden" of real-time perception—especially with high-resolution inputs—creates constraints in terms of processing power, thermal management, and latency. Balancing model accuracy with speed, and ensuring system performance across embedded platforms, is a persistent engineering challenge.
+Location services—often referred to as localization—are essential to autonomous vehicles (AVs), enabling them to determine their precise position within a map or real-world environment. While traditional GPS offers basic positioning, autonomous vehicles require "centimeter-level accuracy," robustness, and real-time responsiveness, all of which present significant challenges.
+One major challenge is the "limited accuracy and reliability of GNSS (Global Navigation Satellite Systems)" such as GPS, especially in urban canyons, tunnels, or areas with dense foliage. Buildings can block or reflect satellite signals, leading to multi-path errors or complete signal loss. While techniques like Real-Time Kinematic (RTK) correction and augmentation via ground stations improve accuracy, these solutions can be expensive, infrastructure-dependent, and still prone to failure in GNSS-denied environments. To compensate, AVs often combine GPS with "sensor-based localization," including LiDAR, cameras, and IMUs (inertial measurement units), which enable map-based and dead-reckoning approaches. Sensor-based dead reckoning using IMUs and odometry can help bridge short GNSS outages, but "drift accumulates over time," and errors can compound, especially during sharp turns, vibrations, or tire slippage. Finally, "map-based localization" depends on the availability of high-definition (HD) maps that include detailed features like lane markings, curbs, and traffic signs. These maps are costly to build and maintain, and they can become outdated quickly due to road changes, construction, or temporary obstructions—leading to mislocalization.
+Path planning in autonomous vehicles is a complex and safety-critical task that involves determining the vehicle's trajectory from its current position to a desired destination while avoiding obstacles, complying with traffic rules, and ensuring passenger comfort. One of the most significant challenges in this area is dealing with dynamic and unpredictable environments. The behavior of other road users—such as pedestrians, cyclists, and human drivers—can be erratic, requiring the planner to continuously adapt in real time. Predicting these agents' intentions is inherently uncertain and often leads to either overly cautious or unsafe behavior if misjudged. Real-time responsiveness is another major constraint. Path planning must be executed with low latency while factoring in a wide range of considerations including traffic laws, road geometry, sensor data, and vehicle dynamics. This requires balancing optimality, safety, and computational efficiency within strict time limits. Additionally, the planner must account for the vehicle’s physical constraints such as turning radius, acceleration, and braking limits, especially in complex maneuvers like unprotected turns or obstacle avoidance. Another persistent challenge is operating with incomplete or noisy information. Sensor occlusion, poor weather, or localization drift can obscure critical details such as road markings, traffic signs, or nearby objects. Planners must therefore make decisions under uncertainty, which adds complexity and risk. Moreover, the vehicle must navigate complex and often-changing road topologies—like roundabouts, construction zones, or temporary detours—where map data may be outdated or ambiguous. Finally, the need for continuous replanning introduces issues of robustness and comfort. The path planning system must frequently adjust trajectories to respond to new inputs, but abrupt changes can degrade ride quality or destabilize the vehicle. All of this must be done while maintaining rigorous safety guarantees, ensuring that every planned path can be verified as collision-free and legally compliant. Developing a system that meets these demands across diverse environments and edge cases remains one of the toughest challenges in achieving fully autonomous driving.
+Algorithms and Methodology for Safety:
+{{:en:safeav:avt:validation.jpg?600|}}
+A major bottleneck remains the inability to fully validate AI behavior, with a need for more rigorous methods to assess completeness, generate targeted test cases, and bound system behavior. Advancements in explainable AI, digital twins, and formal methods are seen as promising paths forward. Additionally, current systems lack scalable abstraction hierarchies—hindering the ability to generalize component-level validation to system-level assurance. To build trust with users and regulators, the industry must also adopt a "progressive safety framework," clearly showing continuous improvement, regression checks during over-the-air (OTA) updates, and lessons learned from real-world failures.
+In terms of "V&V test apparatuses," both virtual and physical tools are emphasized. Virtual environments will play a key role in supporting evolving V&V methodologies, necessitating ongoing work from standards bodies like ASAM. Physical test tracks must evolve to not only replicate real-world scenarios efficiently but also validate the accuracy of their virtual counterparts—envisioned through a “movie set” model that can quickly stage complex scenarios. Another emerging concern is "electromagnetic interference (EMI)," especially due to the widespread use of active sensors. Traditional static EMI testing methods are insufficient, and there is a need for dynamic, programmable EMI testing environments tailored to cyber-physical systems.
+Finally, a rising concern is around cybersecurity in autonomous systems. These systems introduce systemic vulnerabilities that span from hardware to software, necessitating government-level oversight. Key sensor modalities like LiDAR, GPS, and radar are susceptible to spoofing, and detecting such threats is an urgent research priority. The V&V process itself must evolve to minimize exposure to adversarial attacks, effectively treating security as an intrinsic constraint within system validation, not an afterthought.
+Business Models and Supply Chain:
+{{:en:safeav:avt:waymo-robotaxi.jpg?600|}}
+Robo-taxis, or autonomous ride-hailing vehicles, represent a promising use case for autonomous vehicle (AV) technology, with the potential to transform urban mobility by offering on-demand, driverless transportation. Key use models include urban ride-hailing in city centers, first- and last-mile transit to connect riders with public transportation, airport and hotel shuttle services in geofenced areas, and mobility on closed campuses like universities or corporate parks. These models aim to increase vehicle utilization, reduce transportation costs, and offer greater convenience, particularly in environments where human-driver costs are a major factor.  However, the business challenges are substantial. The development and deployment of robo-taxi fleets require enormous capital investment in hardware, software, testing, and infrastructure. Operational costs remain high, particularly in the early stages when human safety drivers, detailed maps, and limited deployment zones are still necessary. Regulatory uncertainty also hampers scalability, with different jurisdictions applying inconsistent safety, insurance, and operational standards. This makes expansion slow and costly.
+In addition, consumer trust in autonomous systems remains fragile. High-profile incidents have raised safety concerns, and many riders may be hesitant to use driverless vehicles, especially in unfamiliar or emergency situations. Infrastructure constraints—such as poor road markings or limited connectivity—further limit the environments in which robo-taxis can operate reliably. Meanwhile, the path to profitability is challenged by competitive fare pricing, fleet maintenance logistics, and integration with broader transportation networks. Overall, while robo-taxis offer significant long-term promise, their success hinges on overcoming a complex mix of technological, regulatory, and business barriers.
+ The evolving economics of the semiconductor industry pose a significant challenge for low-volume markets, where custom chip development is often not cost-effective. As a result, autonomous and safety-critical systems must increasingly rely on Commercial Off-The-Shelf (COTS) components, making it essential to develop methodologies that can ensure security, reliability, and performance using these standardized parts. This shift places greater emphasis on designing systems that are resilient and adaptable, even without custom silicon. Additionally, traditional concerns like field maintainability, lifetime cost, and design-for-supply-chain practices—common in mechanical and industrial engineering—must now be applied to electronics and embedded systems. As electronic components dominate modern products, a more holistic design approach is needed to manage downstream supply chain implications. The trend toward software-defined vehicles reflects this need, promoting deeper integration between hardware and software suppliers. To further enhance supply chain resilience, there's a push to standardize around a smaller set of high-volume chips and embrace flexible, programmable hardware fabrics that integrate digital, analog, and software elements. This architecture shift is key to mitigating supply disruptions and maintaining long-term system viability.  Finally, "maintainability" also implies the availability of in-field repair facilities which must be upgraded to handle autonomy.
-Verification and Validation Capability:
- - Components: Validation of AI components continue to be a stumbling block to overall safety. Continued research is required on methods to mathematically measure completeness, efficiently generate tests, and bound AI behavior. Research directions in areas such as explainable AI, digital twin monitors, and more formal methods may well move the state-of-the-art forward.
- - Scalability: Today, cyber-physical systems do not have a clear hierarchy of abstraction which allows for scaling. Research is required to develop methods where component validation results can lead to higher abstraction validation leading to scale. Learning from the broad methodologies of design artifacts, which enable scaling from semiconductor electronics space, may be good guides for this research direction. 3) Progressive structure: To project trust from consumers and regulators, the industry must develop a clear process which shows a progressive structure of safety progress. A key part of this process are clear arguments for lack of regression of functionality in OTA updates and inclusion of feedback from field failures. Verification and Validation Test Apparatuses 1. Virtual: As methodologies develop for V&V, virtual testing tools will be required to support them. This means that the work done in standards organizations such as ASAM must continue as a key enabling feature to develop these methodologies. 2. Physical: Test tracks must continue to adapt to support their key functions in the overall flow. Specifically, they must provide resources for characterization of virtual models for simulation and be able to quickly recreate complex scenarios from simulation or field failure. Thus, a movie set operational model will be critical for success. 3. EMI: Due to the heavy use of active sensing technologies, EMI is a critical new issue for cyber-physical systems. Today, EMI testing is limited to static testing devices such as anechoic chambers. However, with cyber-physical systems, the combination of mechanical movement and complex reflective materials requires a movement in the state-of-art of EMI testing. Further, today, programmatic methods such as LabView enable efficient test programming for electronics, yet nothing similar exists for cyber-physical EMI testing [48]. Cybersecurity 1. Systematic risk: Networked cyber-physical systems are introducing the notion of systematic risk to bad actors. Governmental oversight is required to ensure that all levels of this chain from physical security to the software architecture are fully dependable. 2. Sensors: Various electronics modalities (e.g., LiDAR, radar, GPS) are critical to autonomous operation while also being vulnerable to spoofing. Research is required to detect and mitigate these spoofing effects. 3. Cyber-security: The surface for cyber-security attacks is constructed through the design and validation process of the product. The V&V process should include a minimization available for adversarial attacks through communication interfaces. This effectively adds a new constraint to the V&V process. Supply Chain: 1. Economics: The economics of semiconductors will limit the development of chips for limited volume markets. Thus, developing methodologies which provide security, reliability and performance from Commercial Off-The-Shelf (COTS) products is critical for success. 2. Design: Field maintainability, skew minimization, and total lifetime cost have been active topics in many industries, but not typically for the electronics components. With the increasing absorption of electronics components, a clear Design for Supply Chain function is required to understand the down-stream costs of electronic design choices in Electronic Design Systems. 3. Supply Chain: In the construction of supply chain relationship, there is often a functional decomposition to suppliers. However, for integrated products, a peek through and joint development of the underlying hardware/software system is highly recommended. Of course, this is exactly the direction taken by the concept of a software defined vehicle. 4. Hardware Fabrics: Building on a minimal number of high -volume semiconductor chips is critical to the long- term viability of the LLC supply chain. To enable this process, a broader concept of hardware programmable fabrics which include software, digital hardware, and analog/sensor function must be developed. Development of this newer version of the computer architecture is key to absorbing the supply chain shocks from the consumer marketplace. Finally, at a systems level there are two strong recommendations: 1. Integrated Functional Approach: V&V, Cybersecurity, and supply chain form the crux of the product assurance function. Thinking of these in an integrated, not siloed, fashion, especially in the process of initial product design, is highly recommended. 2. Learnings from adjacent fields: Automotive is the largest cyber-physical marketplace. The critical learnings from automotive for other ground vehicles as well as adjacent fields such as airborne, space, and marine systems is highly recommended.

en/safeav/avt/challenges.1754092141.txt.gz · Last modified: 2025/08/01 23:49 by rahulrazdan