Modeling and analysis include rigorous mathematical proofs of the proposed theorems and the practical consideration of limited sensing resources and observation costs. A survey of the well-developed coverage control problem is also provided as a foundation of search algorithms within the overall decision-making strategies. Applications in both underwater sampling and space-situational awareness are investigated in detail.
The control strategies proposed in each chapter are followed by illustrative simulation results and analysis. Academic researchers and graduate students from aerospace, robotics, mechanical or electrical engineering backgrounds interested in multi-agent coordination and control, in detection and estimation or in Bayes filtration will find this text of interest. Hussein 2 1. Buy options. The new network we proposed in this paper, J-Net, had about thousand trainable parameters, which was half of our implementation of PilotNet, that had about thousand trainable parameters, and J-Net model had times less parameters than the reimplementation of AlexNet, that had over 41 million trainable parameters, which means that we succeeded to deliver the least computationally demanding solution.
The size of the J-Net trained model was four times smaller than the PilotNet model and about times smaller than the AlexNet model. The smaller size of the network and the number of trainable parameters led to improvement of real-time performance in terms of latency reduction, and to the downsizing of the need for interfacing hardware in terms of computational power, cost, and space.
Based on these results, we can say that our proposed network had less deep architecture than the other solutions we compared it with, a smaller number of trainable parameters, and, consequently, was the smaller trained model. This recommends the designed network for deployment on embedded automotive platforms. The verification of successful autonomous driving was done in the simulator on the representative track. During autonomous driving mode, the signal from the central camera mounted on the vehicle was continuously acquired and sent as an input to the trained machine learning model, which resulted with the control of the steering angle.
Autonomous driving using all three models was recorded and given in the videos in [ 66 , 67 , 68 ] for AlexNet, PilotNet, and J-Net, respectively.
- Sign up today;
- Swipe to navigate through the chapters of this book.
- Conclusion and Future Work | gyqacyxaja.cf;
- A Review and Analysis of Literature on Autonomous Driving.
As can be seen from the given videos, the J-Net fulfilled the requirement for autonomous driving in a predefined path, where the vehicle remained on the road for the full duration of the ride. The measurement of the performance was a successful drive on the representative track, the behavior that the vehicle does not get off the track during the ride, which implies that the better performing solution was the one where the vehicle was in the middle of the track for the full duration of the ride. The performance of the J-Net model was satisfactory.
The qualitative performance evaluation of autonomous driving using implemented networks is given in Table 4. One of the metrics for performance evaluation was observation of the vehicle behavior in curves, Table 4 second row. Among those three solutions, AlexNet performed best during the autonomous driving. Using AlexNet for autonomous driving, the vehicle was in the middle of the road most of the time, while during autonomous driving using PilotNet and J-Net, the vehicle was almost always in the middle of the road, but in some curves it came close to the edge. However, all three implementations of the autonomous driving succeeded to drive the vehicle on the road at all times, and did not go off the path.
Can self-driving cars think ?
In addition to observing autonomous driving on a representative track, the steering angle predictions used for autonomous driving were evaluated. As can be seen in Figure 16 , the steering angle predictions for all three models were relatively similar. The graphical presentation of steering angle predictions used for real-time inference is given for one full lap of autonomous driving on the representative track.
Positive and negative steering angle values represent the left and right angle of rotation. Since the representative track used for driving during inference was the same, and since the speed of the vehicle had been fixed due to simplicity, the steering angles in Figure 16 shows the steering angle predictions in similar frames. Steering angle predictions for the J-Net and PilotNet model were similar; however, the J-Net had a bit higher values, in both directions, left and right.
The AlexNet model resulted with the mostly smooth steering predictions in the majority of the ride.
Conclusion and Future Work
However, in some points, it had extreme values, for example, at about frames there is a spike in the left direction, while the other two models did not have that sharp turn for that part of the road. Steering angle predictions used for autonomous driving, presented as normalized absolute values of steering angle in degrees.
As another measure of autonomous driving performance, we measured the impact of autonomous driving using each neural network on the trajectory. The relative deviation from the center of the trajectory per one full lap of autonomous driving is presented in Figure The driving track characteristics may be seen as four main categories: a mostly straight part of the road rounded with shoulders, curves defined with red and white stripe, bridge, and parts of the road defined without any marks but with dust.
As it can be seen from Figure 17 , driving using all three models had similar patterns. The models show the car driving mostly without oscillations in straight parts of the road. However, in the curves, the deviation from the center of the trajectory was the biggest e. Additionally, Figure 17 shows that all three networks had the deviations in this part, where AlexNet had the biggest deviation, and J-Net performed better than the other models.
On the other hand, J-Net had more oscillations during the full lap, while AlexNet had the best performance, being closest to the center of the trajectory for most of the ride. Relative deviation from the center of the trajectory per one full lap of autonomous driving.
The four main characteristics of the trajectory are parts of the track defined with: a shoulders—regular, mostly straight road; b red and white stripe—mostly sharp curves; c a small wall—the bridge; d red and white stripe and dirt—a sharp curve. Statistical analysis of autonomous driving is also presented through the histograms. This analysis is significant for long term tests. In order to examine oscillations, the histogram of the relative deviations from the center of the trajectory per one full lap of autonomous driving is presented in Figure Histogram of J-Net driving is presented in Figure 18 a, where it was shown that the J-Net has the smallest deviation in the curves, while the oscillations for the center of the trajectory were the biggest.
Histogram of the relative deviation from the center of the trajectory shows that using AlexNet for autonomous driving, Figure 18 c, had the most stable driving experience, with the smallest oscillations from the center of the trajectory. On the other hand, there was an occurrence of sporadic high deviation from the center of trajectory in one curve direction. However, this deviation was within the allowed limits, the car did not come out of the road, which was the criteria that we defined for successful autonomous driving.
Histograms of deviation from the center of the trajectory per one full lap of autonomous driving. Finally, all models performed well, successfully finishing the lap of autonomous driving with no significant deviation from the center of the trajectory. Differences between autonomous driving using different models were notable, but not large.
Based on the computational complexity analysis, it was expected that J-Net would have the least latency and the highest frame rate among the three evaluated solutions. Quantitative performance evaluation verified this claim, as can be seen from Table 5. This evaluation was done on the PC platform explained in Section 6. The J-Net was able to successfully finish the task of autonomous driving on the representative track.
As the track is a closed loop, we measured the number of successful consecutive laps for ten laps in total. All three models were able to successfully drive during the measured time. For the latency measurement, we calculated the time between two consecutive predictions. This number varies during the whole lap of autonomous driving, so for the latency, the mean value was used. The frames per second were calculated by counting the number of predictions per acquired frames in one second.
Quantitative performance evaluation of autonomous driving using AlexNet, PilotNet, and J-Net on a high-performance platform used for the simulator environment. However, if we were using a scalar processor for the inference, the major differences would be expected e.
In the experiment where the simulator environment was used, the inferencing platform was a high-capacity computer with GPU that provided data parallelization. Hence, those results were for the particular application where the GPU was used. Here, since the neural network architectures were more different in their surface area than depth, the majority of operations were able to be done in parallel, so the difference in frame rate was the consequence of the sequencing in the algorithm execution, which was proportional to the network depth.
The faithful demonstration of J-Net performance advantages is platform dependent. If we go to another extreme, when the operations are only done on scalar processors, it is expected that the execution rates would be much different, that is, comparable to the network capacity, the number of parameters.
Real implementations of the J-Net model are intended for the embedded platforms, in which the degree of parallelization will be set so that the performance requirements in frame rate are met. The development of high-performing computers able to perform training and inference for machine learning models leads to great advancement in novel approaches to known problems. However, the industrial application often requires machine learning solutions that can be deployed on computationally inexpensive and smaller memory demanding embedded platforms that have low cost and size.
Deploying machine learning models on a low-performing hardware platform implies usage of an inexpensive models in terms of computational power and memory resources, which can be achieved by careful design of the neural network model architecture. In parallel with advancement in hardware development, in the development of novel processor units targeted for machine learning and, more precisely, deep learning applications, there is a trend in the design of light network architectures that can meet strict hardware requirements.
The deep neural network presented in this paper is one possible solution for end-to-end learning for autonomous driving. The aim of our work was to achieve successful autonomous driving using a light deep neural network that is suitable for inference and deployment on an embedded automotive platform. Having this in mind, we designed and implemented J-Net, a deep convolutional neural network able to successfully perform the task of autonomous driving in a representative track, with the computational cost of using this network being the smallest among other known solutions that have been also explored in this paper.
The main contribution of proposed work is the novel solution that is computationally effective due to relatively light architecture. The complexity of an algorithm is determined by the number of operations in one iteration, and our deep neural network has shown similar qualitative results gained with much fewer operations in comparison with other neural networks explored in this paper.
The possible limitation of J-Net could be the insufficient generalization for the more complex-use case scenarios. In addition to this, our model is trained using raw camera images and steering angle measurements per each frame, while the speed of the vehicle is taken as a constant due to the simplicity. This causes the limitation during autonomous driving regarding the speed since the constant speed is implied. However, it would be possible to train the J-Net to predict the speed of the vehicle. A similar approach to predicting steering angle can be used, which may lead to making simultaneous predictions for steering angle and speed based on the input camera image in real-time.
The future work will include the deployment of the presented network in an embedded automotive platform with limited hardware resources, low processor power, and low memory size. The possible final use cases for the presented end-to-end learning network are robot-cars in warehouses and delivery vehicles. Usage of light DNN solution, like the one presented in this paper, enables deployment on embedded automotive platforms with low-power hardware, low cost, and size, which is important for practical industrial applications.
Conceptualization, J. National Center for Biotechnology Information , U. Journal List Sensors Basel v. Sensors Basel. Published online May 3. Author information Article notes Copyright and License information Disclaimer. Received Mar 15; Accepted Apr Abstract In this paper, one solution for an end-to-end deep neural network for autonomous driving is presented.
Keywords: autonomous driving, camera, convolutional neural network, deep neural network, embedded systems, end-to-end learning, machine learning. Introduction Research and development in the field of machine learning and more precisely deep learning lead to many discoveries and practical applications in different domains. Open in a separate window. Figure 1. Figure 2. Related Work Deep learning is a machine learning paradigm, a part of a broader family of machine learning methods based on learning data representations [ 8 , 9 , 10 , 11 ].
Figure 3. Autonomous Driving System In our approach, we used end-to-end learning for an autonomous driving system. Figure 4. Figure 5. Simulator Environment The platform used for data collection and inference, evaluation of successful autonomous driving was a self-driving car simulator [ 7 ].
Figure 6. Figure 7. Figure 8. Dataset 4. Data Collection Data collection was done while the vehicle was driving in manual mode on the representative track. Figure 9. Data Augmentation For the data acquisition, several laps were recorded while driving in the manual mode, where the data from three cameras were collected. Table 1 Dataset. Data Preprocessing For training the deep neural network, images acquired from all three cameras were used—central, left, and right; Figure 9. Figure Proposed Approach The leading idea during the design process was to achieve end-to-end autonomous driving using the light model computationally least expensive model , while simultaneously achieving the best possible performance, in our case the autonomous driving on the representative path.
The J-Net Architecture Introducing more hidden layers to a deep neural network helps with parameters efficiency. Implementation In order to make an objective performance evaluation of our network, we re-implemented three neural network models: LeNet-5 [ 13 ], AlexNet [ 6 ], and PilotNet [ 1 ], with small modifications in order to be able to aid end-to-end learning for autonomous driving.
Design Details for AlexNet and PilotNet Re-Implementations In our work, the AlexNet architecture [ 6 ] was re-implemented and adapted for the purpose of end-to-end learning for autonomous driving. Results and Discussion The proposed deep neural network J-Net was compared with AlexNet and PilotNet, which we re-implemented in order to conduct an objective performance evaluation of the novel design. Computational Complexity The execution of the deep neural network models depends heavily on a set of static constants, weights, also called model parameters.
Performance Evaluation during Inference—Autonomous Driving The verification of successful autonomous driving was done in the simulator on the representative track. Table 5 Quantitative performance evaluation of autonomous driving using AlexNet, PilotNet, and J-Net on a high-performance platform used for the simulator environment. Conclusions The development of high-performing computers able to perform training and inference for machine learning models leads to great advancement in novel approaches to known problems.
Author Contributions Conceptualization, J. Conflicts of Interest The authors declare no conflicts of interest. References 1. Bojarski M. End to end learning for self-driving cars. Explaining how a deep neural network trained with end-to-end learning steers a car. Mehta A. Learning end-to-end autonomous driving using guided auxiliary supervision. Chen Y. Ramezani Dooraki A. Krizhevsky A.
Imagenet classification with deep convolutional neural networks. In: Pereira F. Advances in Neural Information Processing Systems. Udacity, Inc. Self-Driving Car Simulator. Goodfellow I. Deep Learning. Aggarwal C. Neural Networks and Deep Learning. Springer International Publishing; Cham, Switzerland: Chollet F.
Deep Learning with Python. Sutton R. Reinforcement Learning. LeCun Y. Backpropagation applied to handwritten zip code recognition.
Search and Classification Using Multiple Autonomous Vehicles | SpringerLink
Neural Comput. Simard D. Best practices for convolutional neural networks applied to visual document analysis; Proceedings of the Seventh International Conference on Document Analysis and Recognition; Edinburgh, UK. Shin H. IEEE Trans. Pathak D. Karpathy A. Chi J. Remote Sens. Russakovsky O. Imagenet large scale visual recognition challenge. Simonyan K. Very deep convolutional networks for large-scale image recognition. Szegedy C. Visin F. Renet: A recurrent neural network based alternative to convolutional networks. Zoph B. Acuna D. Wang T.
Silver D. Mastering the game of go without human knowledge. Mastering chess and shogi by self-play with a general reinforcement learning algorithm. Amodei D. Chen Z. Yao Y. Kanade T. Wallace R. Dickmanns E. IFAC Proc. Thrun S.
Field Robot. Montemerlo M. Buehler M. Springer Tracts in Advanced Robotics. IEEE Sens. Chavez-Garcia R. Cho H. Ravankar A. Wei K. Cai W. Paden B. Sung Y. This machine learning technique consists of learning from experience. If we want to turn right, we ask the car to make a random choice, if it is good, it receives a positive reward, if not, a negative. Over the course of the training, the car is able to learn what has caused a positive reward, and to reproduce it. This technology is at this day the one that comes closest to human learning.
It is possible to integrate this technology in robotics and especially in autonomous vehicles. The MobilEye planning system works with reinforcement learning. The recent demo of Wayve perfectly demonstrates the use of this concept. As part of my Nanodegree on autonomous vehicles, I realized a project on highway driving. I had to develop an algorithm that could drive 7. The Finite-State Machine introduced earlier is used to overtake a slow vehicle or slow down if overtaking is not possible. Autonomous navigation is an exciting subject.
We drive with our intuition and our eyes respecting the rules of the road. To reproduce this in a computer, you have to go through all the steps that we have done. We must see, position ourselves, predict the behavior of other vehicles, and finally make a decision by integrating constraints such as the law or the comfort of passengers. Behind the machine, a human tells what are the actions we must priviledge in certains situations. This subject leaves room for a very large number of research and experimentation works.
It allows to reach the level 5 of autonomy and democratize permanently the arrival of self-driving vehicles. Discover this article in French. Next on Controllers and Final Integration. Sign in. Get started. Can self-driving cars think? Jeremy Cohen Follow. Autonomous Navigation. Trajectory Generation. Path Planning We have just studied the generation of low-level trajectories. Results As part of my Nanodegree on autonomous vehicles, I realized a project on highway driving.
Conclusion Autonomous navigation is an exciting subject. Jeremy Cohen. Towards Data Science Sharing concepts, ideas, and codes. Towards Data Science Follow. Sharing concepts, ideas, and codes. Write the first response. Discover Medium. Make Medium yours.
- Automated Vehicles for Safety | NHTSA.
- The Machine Learning Algorithms Used in Self-Driving Cars.
- Guru Tattva.
- Autonomous Vehicles: The Mix of Opportunity and Uncertainty | FierceElectronics.
- The Healing Remedies Sourcebook: Over 1000 Natural Remedies to Prevent and Cure Common Ailments;
Become a member. About Help Legal.