(49-6) 28 * << * >> * Russian * English * Content * All Issues

Bulk cargo volume measurement for moving dump trucks with a single-layer LiDAR and a camera
D.A. Bocharov 1,2, V.V. Kokhan 1, I.D. Konyushenko 1,3, A.Y. Resniansky 2, I.P. Nikolaev 1,2, D.P. Nikolaev 2,4

1 Institute for Information Transmission Problems,
127051, Russia, Moscow, Bolshoy Karetny per. 19, build. 1;
2 Federal Research Center "Computer Science and Control", Russian Academy of Sciences,
119333, Russia, Moscow, Vavilova 44, build. 2;
3 Moscow Institute of Physics and Technology,
141701, Russia, Dolgoprudny, Institutskiy per. 9;
4 Smart Engines Service LLC,
117312, Russia, Moscow, pr. 60-letiya Oktyabrya 9

 PDF, 1816 kB

DOI: 10.18287/COJ1801

Pages: 1120-1128.

Full text of article: English language.

Abstract:
The paper addresses the problem of non–contact bulk cargo volume estimation for moving dump trucks. A common scanning method that lets to evaluate the volume of cargo of complex surface for a moving truck implies two single-layer (2D) Light Detection and Ranging (LiDAR) sensors: one is used to scan a vehicle in a plane perpendicular to its movement and the second – to estimate vehicle displacements and restore scans positions on an axis along vehicle movement direction. While LiDAR sensors provide reliable measurement signals in controlled environments their efficacy drastically decreases under challenging outdoor conditions: sand dust, fog, rain heavy precipitation cause false detections and distort LiDARs signal. Thus, vehicle displacements estimated with a highly corrupted LiDAR signal can not be used for a reliable measurement as they may lead to significant volume calculation errors. Partially this is solved in multi-echo lidar where distorted data could be separated from the relevant. In contrast to the single-echo 2D LiDAR, image data from industrial cameras is less sensitive to sand dust or fog. In the paper we propose a novel bulk cargo estimation method that implies only one 2D LiDAR and for vehicle displacements estimation utilizes a camera and computer vision methods. As we demonstrate on a diverse dataset of 730 pairs of dump truck passes from an operating sand pit, the proposed method is more accurate than the two 2D LiDARs baseline while requiring a significantly cheaper sensor. In case if a camera is already present in the volume measurement system and utilized for loaded material classification then the proposed method lets to reduce the cost of solution by the cost of one lidar.

Keywords:
bulk cargo, dump trucks, volume measurement, LiDAR, camera, optical flow, deep learning, outdoor, sand.

Citation:
Bocharov DA, Kokhan VV, Konyushenko ID, Resniansky AY, Nikolaev IP, Nikolaev DP. Bulk cargo volume measurement for moving dump trucks with a single–layer LiDAR and a camera. Computer Optics 2025; 49(6): 1120-1128. DOI: 10.18287/COJ1801.

References:
  1. Niskanen I, Duan G, Suzuki H, Endou D, Immonen M, Hiltunen M, Yamauchi G, Tyni P, Hashimoto T, Heikkilä R. Determining Payload on Platform of Lorry in Real Time Using Integrated 3–D Lidar From Excavator Boom. IEEE Transactions on Instrumentation and Measurement; 2024 Jan 5;73:1–7. DOI 10.1109/TIM.2024.3350119.
  2. Amorim LL, Mutz F, De Souza AF, Badue C, Oliveira–Santos T. Simple and Effective Load Volume Estimation in Moving Trucks using LiDARs. 2019 32nd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), Rio de Janeiro, Brazil: IEEE; 2019, p. 210–7. DOI: 10.1109/SIBGRAPI.2019.00036.
  3. Liu Y, Tao X, Li X, Colombo AW, Hu S. Artificial intelligence in smart logistics cyber–physical systems: State–of–the–arts and potential applications. IEEE Transactions on industrial cyber–physical systems. 2023 Jun 6;1:1–20. DOI: 10.1109/TICPS.2023.3283230. DOI: 10.1109/TICPS.2023.3283230.
  4. Sritap C, Pitayachaval P, Tantrairatn S. Development of a bulk material volume estimation system using automatic moving rail LiDAR technology. International Journal of Applied Methods in Electronics and Computers. 2024 Jun 30;12(2):48–53.
  5. Duff E, Automated volume estimation of haul–truck loads, Proceedings of the Australian Conference on Robotics and Automation. 2000, p. 179–184, CSIRO. DOI: 10.58190/ijamec.2024.97.
  6. Grigoryev A, Khanipov T, Koptelov I, Bocharov D, Postnikov V, Nikolaev D. Building a robust vehicle detection and classification module. InEighth International Conference on Machine Vision (ICMV 2015) 2015 Dec 8 (Vol. 9875, pp. 313–319). SPIE. DOI: 10.1117/12.2228806.
  7. Kokhan VV, Konyushenko ID, Bocharov DA, Seleznev IO, Nikolaev IP, Nikolaev DP. TSQ–2024: a categorized dataset of 2D LiDAR images of moving dump trucks in various environment conditions. InSeventeenth International Conference on Machine Vision (ICMV 2024) 2025 Feb 24 (Vol. 13517, pp. 60–65). SPIE. DOI: 10.1117/12.3055203.
  8. Zhang Z, Gupta A, Jiang H, Singh H. Neuflow v2: High–efficiency optical flow estimation on edge devices. arXiv preprint arXiv:2408.10161. 2024 Aug 19. DOI: 10.48550/arXiv.2408.10161.
  9. C. E. Sung, K. Young–Jin, J. Y. Joon, and J. S. Chan, Extraction of loaded cargo and volume calculation pcd for cargo vehicles. ICIC Express Letters Part B: Applications 2023 14(09), 977. DOI: 10.24507/icicelb.14.09.977.
  10. Savinetskij AB, Evstigneev VE, Kazaryan SM, Chudnikov VV, Bychkov AV, Rudzitis A, Kovalev AV. Method for determining the volume of bulk cargo in a moving vehicle using non–contact measurement. Russian Patent RU 2772138C1 of May 18, 2022.
  11. Liu Q, Feng C, Song Z, Louis J, Zhou J. Deep Learning Model Comparison for Vision–Based Classification of Full/Empty–Load Trucks in Earthmoving Operations. Applied Sciences 2019;9:4871. DOI: 10.3390/app922487.
  12. Sun X, Li X, Xiao D, Chen Y, Wang B. A method of mining truck loading volume detection based on deep learning and image recognition. Sensors. 2021 Jan 18;21(2):635. DOI: 10.3390/s21020635.
  13. Deniz M, Alam F, Yuan C, Ko HS, Lee HF. Single image based volume estimation for dump trucks in earthmoving using machine learning approach. EPiC Ser. Built Environ.. 2022 May;3:380–8.
  14. Kwak J, Bae J, Huh J, Moon DL, Hong D. Vision–based Payload Volume Estimation for Automatic Loading. Proc. of the 9th Intl. Conf. of Asian Society for Precision Engg. and Nanotechnology (ASPEN 2022) (2022). DOI: 10.3850/978–981–18–6021–8 OR–02–0280.
  15. Chen J, Lu W, Yuan L, Wu Y, Xue F. Estimating construction waste truck payload volume using monocular vision. Resources, Conservation and Recycling. 2022 Feb 1;177:106013. DOI: 10.1016/j.resconrec.2021.106013.
  16. Al–Qudah S, Yang M. Large displacement detection using improved lucas–kanade optical flow. Sensors. 2023 Mar 15;23(6):3152. DOI: 10.3390/s23063152.
  17. Lowe DG. Distinctive image features from scale–invariant keypoints. International journal of computer vision. 2004 Nov;60(2):91–110. DOI: 10.1023/B:VISI.0000029664.99615.94.
  18. Rublee E, Rabaud V, Konolige K, Bradski G. ORB: An efficient alternative to SIFT or SURF. In2011 International conference on computer vision 2011 Nov 6 (pp. 2564–2571). Ieee. DOI: 10.1109/ICCV.2011.6126544.
  19. Tan D, Liu JJ, Chen X, Chen C, Zhang R, Shen Y, Ding S, Ji R. Eco–tr: Efficient correspondences finding via coarse–to–fine refinement. InEuropean Conference on Computer Vision 2022 Oct 23 (pp. 317–334). Cham: Springer Nature Switzerland. DOI: 10.1007/978–3–031–20080–9_19.
  20. Deng Z, Yao Y, Deng B, Zhang J. A robust loss for point cloud registration. InProceedings of the IEEE/CVF international conference on computer vision 2021 (pp. 6138–6147).
  21. Brightman N, Fan L. A brief overview of the current state, challenging issues and future directions of point cloud registration. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences. 2022 Oct 27;10:17–23. DOI: 10.5194/isprs–annals–X–3–W1–2022–17–2022
  22. Sidorchuk DS, Pavlova MA, Kushchev DO, Selyugin MA, Nikolaev IP, Bocharov DA. CADCP: a method for chromatic haze compensation on remotely sensed images. InSixteenth International Conference on Machine Vision (ICMV 2023) 2024 Apr 3 (Vol. 13072, pp. 346–355). SPIE. DOI: 10.1117/12.3023507.
  23. Filin AI, Kopylov AV, Gracheva IA. Method for removing haze from images, captured under a wide range of lighting conditions. Computer Optics. 2024;48(1):102–8. DOI: 10.18287/2412–6179–CO–1361.
  24. Volkov VV, Shvets EA. Neural network algorithm for optical–SAR image registration based on a uniform grid of points. Computer Optics. 2024;48(4):610–8. DOI: 10.18287/2412–6179–CO–1426.

© 2009, IPSI RAS
151, Molodogvardeiskaya str., Samara, 443001, Russia; E-mail: journal@computeroptics.ru ; Tel: +7 (846) 242-41-24 (Executive secretary), +7 (846) 332-56-22 (Issuing editor), Fax: +7 (846) 332-56-20