王飞跃的个人博客分享 http://blog.sciencenet.cn/u/王飞跃

博文

端对端平行无人矿山系统及其关键技术

已有 11015 次阅读 2019-12-24 08:48 |系统分类:论文交流|关键词:学者

 端对端行无矿山系及其键技术

 

杨超,高 云峰 陈龙 飞跃

 

【摘 】针对矿山无人化的切需求与研究现状,提出了基于 ACP 方法的端对端平行无人矿山系统解决方案。 首先分析了无人矿山运输系统的研究现状设计了平行无人矿山的基本框架提出了平行无人矿山运输系统的主 要模块包括无人矿山管控中心无人矿卡运输平台半自主挖掘/铲运平台和远程接管平台 4 部分并探讨了实现平行无人矿山系统的 7 关键技术即虚拟端平行矿山构建矿山环境感知与定位矿山设备的单机无人控 多设备之间的协同作业平行矿山管控平行接管决策以及矿山网络通信平行无人矿山系统的实施与部署将有效提高矿山企业运作效率及安全生产水平,推进绿色矿山建设,对实现可持续发展具有重要意义。

 

【关键】平行无人矿山;管控中心;接管决策;协同作业;无人矿卡;半自主挖/铲运

 

引用格式 杨超, 高玉, 艾云峰, 田滨,陈龙, 王健, 王飞跃. 端对端平行无人矿山系统及其关键技术. 智能科学与技术学报[J], 2019, 1(3): 228-240

 

End-to-end parallel autonomous mining systems and key technologies

 

 

YANG Chao, GAO Yu, AI Yunfeng, TIAN Bin, CHEN Long, WANG Jian, WANG Fei-Yue

   

Abstract In view of the current research situation and the urgent demand for unmanned mines, the end-to-end parallel autonomous mining solution was proposed based on the ACP method in this paper. The research status of unmanned mining transportation system was analyzed. The basic framework of parallel autonomous mining was designed. The par- allel autonomous mining are mainly composed of four parts: the management and control center for autonomous mining, the autonomous transportation platform of truck, the autonomous mining/shovel platform and the remote take-over plat- form. The 7 key technologies for realizing parallel autonomous mining system were discussed, namely, the virtual paral- lel mine construction, environment perception and localization, unmanned control for single equipment, collaborative work between multi-equipment, management and control for all the equipment, decision for parallel taking over, and mine network communication. The implementation and deployment of parallel autonomous mining systems will effec- tively improve the operational efficiency and the level of safe production of mining enterprises, and will promote the construction of green mines, which is of great significance for achieving sustainable development.

 

Key words parallel autonomous mining, management and control center, take-over decision, collaborative work, au- tonomous truck, autonomous mining/shovel

 

Citation YANG Chao. End-to-end parallel autonomous mining systems and key technologies. Chinese Journal of Intelligent Science and Technology[J], 2019, 1(3): 228-240 
 
  

1    引言

 

作为术不 仅是转战大互联网共道路开临诸交通 标志知的稳定 的硬设备人车走向市道 。因中低车将 是无并商 山系驾驶 相关之一。


矿山系统是一个典型的复杂系统,它包括生产(开采)子系统、调度子系统、供水供电子系统、 运输子系统、设备及人员管理子系统、安全子系统 等各个方面。要实现矿山系统的无人化,生产、运 输、调度、安全等子系统是关键[1]


然而受矿山地理位置偏僻、生活环境较为 影响步及 人们天矿日益 凸显招聘 难度煤矿上的 其他方面带来 的安质上安全 风险矿区高, 对矿商开 发和矿山美国 卡特公司实现 了无自动俄罗 斯的维克出了无人使自动占比例还程机公司 合作方案的是卡特彼勒公公司


在矿卡需求方面,目前全球矿卡每年销售总量(不包括管理系统)在 1 000 台左右,总采购成本大约为 20 亿美元。预计到 2020 年,澳洲、欧洲和美洲的主要矿业公司中 50%将使用无人化产品,部分 矿山已经决定全部采用无人矿卡。但目前无人矿卡 的占比仅不到 5%,主要由卡特彼勒公司和小松公司两家矿卡生产商集中供应。


对于矿山企业,实现矿山无人化具备以下优势:增减少降低劳动安全外; 降低受极少; 便于


具备 给用带来成本 、环相应地,辆及的盈利,无人;在常规无人矿卡整套方案监控与调 系统。除上,无人矿促进修等后市 收入的进一步提高。


2    无人矿山研究现状

 

早在 20 90 数字就已经出采矿式上不断这种矿山作业境安基础上实矿山持续性安全远距 离测仿等技地质 勘测和矿以及相关据的 记录实现 科学可靠[2]


为实小松自动 系统(AHS)用于矿山队的运营和管理[3] 此系都配高精 度定系统线 车队定运根据 接收的自 动运人卡人员和车2018 10 公司 宣布 AHS (包人卡 车)已实 20 亿矿石运输[4]


工程机械巨头卡特彼勒公司亦是矿卡自动化领域,它 2004 展出了矿MineGEM[5]使操作人员危险之外或者程控采矿设性的管理 系统彼勒公山之星(Cat MineStar), Fleet、地形(Terrain、检测(Detect)与指令Command)套件。性能套件为全集成,所有信 息可于优安全 提升机器的利用率及工作时间卡特彼勒公司 称其综合性采矿作备管理系统已实 10 亿 矿石输。


山特维克也是世界上较早开发自动化采矿技术的运机下矿山应 20 年零。为证明地上巷道中对进行


2016  年秋天,沃尔沃的无人驾驶卡车在瑞 北部矿区 Boliden 地下 1 320 m 进行测试, 并真使自动环境 中的输环


我国 20 已经取得存在体现在 5 息化繁多的系交叉一个能搭开放件投入及统之融合分析无法顺利开展,产生大量信息孤岛;管理系统护成[6]


综上所述,国外企业像卡特彼勒公司、小松公司、日立、沃尔沃等均已实现了不同程度的矿山无人化运作,我国在无人矿山的进程方面也在不断加快步伐,整体上已经具备在单台矿用卡车上做无人驾驶的能力,但在设备间的协同作业以及集群化运营方面,还需要更多的研究,以实现对无人矿山 安全和效率的提升。更重要的是,我国无人化矿山技术缺乏足够的理论支撑,在知识产权等方面 受限于国外。


为此实矿应的 动态人工系统(即平行矿山系统,利用这个人工系统实时线模拟 计算演化有助于实精准工生、人社会以及计算实等研 奠定础。本文利 用现网技术、云术、 线通信技术等,采集矿山的各个关键系统的实 数据中心以上 数据相对并通过对工系实现对现系统 估、山的理。


3    平行无人矿山整体框架

3.1   ACP 简介

2004 统的思想真实 系统统,理、优化和 引导[7]统的方法 ACP 方法其中 A 人工社会机理 一致C 是计人工 系统效、稳健的学习等方来对同复杂问进行 分析P 用人实系 统,升人系统[8-9]


    1.png


平行系统 ACP 的基本框架如 1 在人真实 系统够准获得 虚拟制和能对真实系统 方式 来描在平 行执升的[10]


计算实验是在虚拟系统和真实系统两个维度中同来越体来说实验前的数据能够行的知识计算为通过预学习习来获取来的 过的使系统[11-12]


平行平行包括虚间的模型流流向通过大学习识,用以是真系统的描能够整个系统对虚行引导和修改[13-15]


3.2    化运分析

ACP 方法的平行无人矿山框架如 2 它是平行系统在矿山领域的一个切实应用平行无人 矿山中主要包 6 个重要的部件(地,分别是采 矿点A、卸矿点(B、传输带卸矿点C、货 火车站D、管控中心(E)以及虚拟平行矿山系F。其 ABC D 可以看作真实系统的组成 部分, E F 以看作虚拟系的组成部分。


在平研究和解 4 方面 的问是在 A 主挖 掘作作业第二个是 A 点到 B 卡运矿卡中加实现 无人第三个是 C D 重型卡相车的 运行的,可自动输、自动卸载(径的划、 避障平行所有 设备控中与调 ,同通信制。 四大够实的无 人化的安率。


3.3    优势

我国在矿山智能化和无人化的建设方面仍然存在巨大的系行理 ACP 将为矿山良好方案[16]利用平行论和 ACP 法建山系统是个高智能化系,其包括备的人力成本本、设路面矿山系统操作管控能力群管协同效率减少损坏导致

 

3.png

 

平行子系一个 与之将现统映入虚,通网、 调度解决 各子调度


将平山系为驱动互动组织 及机山系验对 各类度、协理、制、等问题标是下矿山系矿山水平 与智度。


利用 ACP 行矿基本 思想系统如平行矿卡、平机、平行管理、平信等子系身局统优 化相行子系统优的统。


平行测难 的矿复杂功能为单拆分 子系山系分;同时验和将在 虚拟界中的预测与理策 实矿使系统工作场景

 

4   平行无人矿山系统及功能设计

 

平行山管 控中运输/平台 以及无人责对 无人控与所建立的的实化, 保障安全运输平台挖掘/平台设备 实现装载卸载 一体台负的生 产运台或挖掘/平台主或在由平无人中心的综决策令,远程实现对多设备 管。 3


2.png


4.1   

平行无人矿山管控中心是矿山体系的神经中枢, ACP 契合与控制中管理,实现调度与维护全生产、营。


平行矿山管控中心包括虚拟矿山系统与矿 计算系统 进行三维仿立虚拟矿统,与虚实化、平行执的预优化 无人控中 心平仿备、工控机监控 台、图器、备等。 项设相对发生 危险系统其他 系统员安中心 将做的大监测 矿山与控

 

4.2    

无人括短(无人宽自卸长途运输辆(。所搭载的主要传感器包括激光雷达、毫米 雷达系统相机V2X(车 网)设备、工机、无线网试仪、车远程监控等无人矿卡由控制中心管理控制 为每辆车指定运输路线车辆通过接收无线指令 以合适的速度按照目标路线运根据行驶路线 自身息自载、 运输环。


无人动驾换、 状态理与等模 块。人矿台的感知系统 、毫达、工机感息, 同惯位信的决策规统提信子 系统通与管控心、程接管台的无线通信 实现管控中心的远程监控,以及与运输平台和挖掘/ 铲运据管矿卡 的运便数人机界面过车呈现无人卡的 车辆口,全。考虑到 矿山台还 包括式切检测 子系康状系统 综合控中台的 数据在人式间选择换。


4.3   挖掘/平台

半自/挖掘机等 通过压力达、惯性毫米可实现在与铲运输 平台业。


在挖掘机所在置实区域内矿行驶 规划行驶行装 各类对目 标矿动卸过程 机手可根 360 景影 姿化辅掘过 程进要时既保 证了进行了驾 驶员


而在业中机移 动更便控中目标位置责矿按照 规划至矿后向矿卡物。


4.4    

全性人工可实现对人矿卡运输平台和半自主挖 /铲运平台在特的远其他紧急无人平台发送 工接管请求,切换到人工模[17],远程控制无 矿卡运输平台和半自主挖/铲运平台远程接 障等处理方式 换与策略。一 台可卡或者挖 /装载机的接管操控并且可以通过平行无人的监远程接管协同全与作业同时远程接管也保障了驾驶员的人身安 并提高了工作效率。


5    平行无人矿山的关键技术

 

技术建、、矿山设控制作业、平平行络通信,体说明如下。


5.1    构建

构建平行矿山系统的关键是利用信息技术 各个性进行数现代个采集点 数字视化技术 现实将各采集 导入使统成为与 应的统中利用 、优作业进行 并将,从而实 的优矿山系统 构如 4 所示。


4.png


在图 4 过程据被 传输信息据融合手段山运行系预测矿 3 件定统, 可以矿山[18]。这样, 一方虚拟山生产作行、在线、实仿方面 通过与真互, 完成评估


实际杂多其建 立可矿山确控 实现的多的动景和实体域均的应用成前已虚拟安全统、车辆 统、三维地震视化分析系机远用系统[19]随着断进将在矿井风险评价安全 数据可化直观现以各类综应用分等方 提供更完善的决方。矿山字信息平行 统的建,可将杂的际采矿统转变与多平行子统相结的智系统,实际状进行踪和测,通过算实和优化为具体案的定提供靠的科依据平行系的建立以有 预估生过程的展方,并为同情况供优 解决方,大大减少事发生的能性,矿山 全作业具有重要的具有广阔的用前景。

 

5.2   定位

5.2.1  感知

环境(激米波 相机传感 主要及可动静 态障标识态监测等下,路况更杂,障种类 多,可括:空旷区域,路起伏、 簸,现较块、树木等,路面为陡峭的坡路等。矿山机械在进 作业合技进行 检测并在域内对障物进跟踪标进碰撞 预警


检测可行碍物等信导航对于路面可采传感器提保证机械 GPS 位失的情沿着可行区域目标点,通达、相机等传感测出对动 态障知信块, 从而避障能。


统的、行挡墙等。 包括深度学习其中有支持向量机SVMBoosting最近邻分类器等基于度学习的方法有神经网络卷积神经网CNN)、 全卷积神经网络FCNR-CNNFast R-CNN RPN 等。通过大量的样本特征对网络进行训练 得到网络参数。


5.2.2   定位

矿用设备的定位技术有航迹推算定位技术 线技术、视觉定推算确定一初备的传感器推体位线技术是指线的直线过测量固体的。无线电无线线系统和全 。地预先获得 图信过程中, 载的取设备周 信息地图信息 获取。视觉定 术是指过电耦合器件(CCD或其他视 周围对环境中设定 识别和 定标进而得到备的纵向位置信息。

 

矿山外环境缺和参线定; 路面路面够,无法使用光辆起比较 大等更高定位方式定位。


•    于矿山应点, 可采用基差分定性导航系RTK-INS 和激LIDAR-SLAM 部分RTK-INS 能够完级别 的定体和卫星 系统GNSS时动态real-time kinematic RTK,需 LIDAR-SLAM 进行辅 助定


对于地下定位,可采用激光站定位超宽UWB的定位方本可 10 cm 以内 [20]可以步定 建图SLAM方法行定位激光定统利激光器与目标间的 服控机构所转角度计算 的坐标,实现对地下采矿设的定位。采用 UWB 定位术研采矿设备确定两部分在巷节点,这类节点在定位系统中会预先设置好绝 坐标行定部分是系装在通过 无线定动过锚 节点动节信息交互出地备的精


5.3    控制

矿山设备(卸车车、 掘机等)不常规辆,它们本身的采用也存 在一山移大、 系统大、载等众点。此矿山 工地面条在大 量连面不无论 是整辆更 加复使制难制算 法具稳健性。


通过态检息, 单机望轨速度

的控和卸其子 系统系如 5 示。


 
   5.png

 
 
  

各部以下面。

•    横向 度组的设定的 理想驶轨跟踪控制法对 行控为全式差 速转一定此外, 矿山备定 算法跟踪 控制度。

基于控制[21]和基 地形场的转向[22-23]具有克服上述缺的先 优势可以的驾 驶经对支系统 的传转向 控制法在控制中具较好性和稳性。


矿山设备行走中的纵向控制相比常规乘 车辆动矿制动 踏板者在替发 挥作,因控制过程需要 进设由于/制动 力不,在下坡前必将速 ,否控,险。由于加 速和障碍须保 持足,才外,还有况的题。


挖掘和铲装过程主要是以规划、定位、 知子统的为基础,过数 法对行计 算和压缸相应 的执掘机通过 与运主装


卸载控制需结合车辆定位和环境感知子 过控路, 实现准确对货 箱内 尚未待或快卸 载剩回位 控制载时


5.4   

在实现矿物从地面到 输设备的转移过 掘机进行密切确迅以挖掘机作业装载区域 入场驶路径需不断进行规划。


装载同作如图 6 所示的路径规划、场、车、 载、程说


 
  6.png

 

首先处的卡装 载时 L,之后过对的感 知情何特车起 始点 S标点 IO 为装和离 ISLO 的坐息, 结合感知载区域矿划。


为保效进要挖掘机需要以及管控进行在入场之待, I 点入场 S L 点停车等装运 L 点后成挖掘机将铲斗矿卡续装卡对自身况进结果通知动作的位置。中,传感器实检测货箱内矿物的装载状态通过快速三维重建 箱内导挖掘机矿物装载。完成矿物装载后, L 点开始离场,定道线要管控中控制 ISSLLO 3 条线路上至少各有一辆时各交互以防止发生碰撞。


5.5   

平行矿山的管控中心主要体现为 2 E 构建的虚 拟矿F 模拟导和 提升如图 7 心主要实三大一是矿卡挖掘调度障监权限 的远


7.png

 

在矿度过中心操作主要挖掘机与辆矿合作权多台与运输模员的就能实运输


整个系统的故障监测过程主要包括挖掘机故障管和卡的异常况报操作车的急停操作监管便操作来保稳定。


在远程遥控监管过程中有经验的驾驶员可以 出远程操控的请求通过中心处理器的识别和下发到特定的需要被控的车辆中从而利驾驶模拟器对远程车辆进行操控这样做的目主要是在车辆异常的情况下通过人的操作来证车辆的安全性从而提升整个智能矿山系统稳健性。


5.6    

远程接管平台用于在故障或者紧急情况下 无人统将 会涉驾驶 员也程接的设 备数监管保证设备监管成人员、成接管原则 说明下。


型进行决 设备个人备, 但只


个人设备 数量少。


当出现需要接管的设备过多而无法全部 管时管的车,

以保


•    度的接管余。


平行策流程如 8 示, 具体下。


 8.png

 

否发否出现控制偏过大的急情来判断否需要管;需要远程接管姿偏差信号故障车速 加速度参数计设备接管需指数;过比所有需接管车的需指数大,确定个设的接管先等级在所监控车的接管求及管等级定后,可对要被接控制的辆进接管对于最接管等的车pi =1直接 监控画和控制切换接管员行接管制; 此时还其他车同时求接管则首先要判 是否还有其他接管车辆的被接管等 pi 值小于或等于接管员总数 m,则将其监控画面和控 权分配其中的个接员控制否则,明没 剩余的员进行管,时必须接管等低的辆进行制动/停车控制,确保车辆安全。当有接管完成车接管而闲后即可对人员不而无 接管的车辆继续进,直到完成所有接


5.7    通信

矿山广计和选择信设方式,保信息全性靠性。遮挡、空气含水率、起的信线质量造成于传 Wi-Fi在网时, 用户UDP分组能达到数无线Mesh)网 随着加,分、时加,另外并需敛, 进而使的概加。


平行矿山的网络通信分为设备与中心之间的远程端管 9 所示协同Wi-Fi Mesh 通信侧、以实 到车(V2V信。客端(CPE通信设备 安装 4G 实现到服V2Server信。


Wi-Fi 链路中的策略:通过部署多通信系统、 链路预算冗余、加大频带和带宽的预算余量等,增强通信系统的可靠性,提高系统的抗干扰能力。结合各种通信协议的特点,根据实际场景和应用情况的不同,选取合适的网络传输协议。


异常处理机制:在网络通信中不可避免地会 存在通信异常情况,如用户数据报协议(SUDP)发生分组丢失、车辆与控制中心的通信断开等。设计完善的异常处理机制,可以有效降低网络异常造成的各种影响[24]。异常处理方式有:将分组丢失情况告知管理员,并记录事件和日志;在通信断开时,及时向控制中心告警以通知管理员,或直接停车以确保安全。


 9.png


 

6    结束语

 

在无外先制造商已山无虽然在无取得但与国外仍存待解决人化够的 理论方面此, 本文 ACP 了端山系统的基优势 进行无人详细论矿山 7 各个关键了说建设提供


在智化的实现使用误操作以动成增强车与能源率; 独创能随机等部件行,延寿,减降低保养的集幅提高矿率。

 

参考

 

[1]    左敏. 管理中的[J]. , 2011, 40(9): 131-133.

ZUO M. The application of parallel system theory in modern mine management [J]. Metal Mine, 2011, 40(9): 131-133.

[2]    高磊.  基于平行系统理论的矿山安全作业系统的建立[J].  金属矿山, 2013, 42(1): 113-115.

GAO L. Security operating system of mine safety based on the parallel systems theory[J]. Metal Mine, 2013, 42(1): 113-115.

[3]    苏子. 或将来临[J]. , 2015.

SU Z M. Construction machinery unmanned era is coming[J]. Con- struction Machinery, 2015.

[4]    How Komatsus autonomous haul trucks work and what it takes to imple- ment the technology at a working mine[N]. Equipment World News.

[5]    EDNIE H. Automation for the underground [J]. CIM Bulletin, 2005, 1085(98): 12-13

[6]    陈晓晶,  .  智慧矿建设架构系及其关键技术[J]. 煤炭科学技术, 2018, 46(2): 208-212, 236.

CHEN X J, HE M. Framework system and key technology of intelli- gent mine construction[J]. Coal Science and Technology, 2018, 46(2): 208-212, 236.

[7]    王飞跃.  关于复杂系统研究的计算理论与方法[J].  中国基础科学, 2004, 6(5): 3-10.

WANG F Y. Computational theory and method on complex system[J]. China Basic Science, 2004, 6(5): 3-10.

[8] WANG F Y, ZHENG N N, CAO D P, et al. Parallel driving in CPSS: a unified approach for transport automation and vehicle intelligence[J]. IEEE/CAA Journal of Automatica Sinica, 2017, 4(4): 577-587.

[9]    WANG F Y. The emergence of intelligent enterprises: from CPS to CPSS [J]. IEEE Intelligent Systems, 2010, 25(4): 85-88.

[10]  王飞跃.  平行系统方法与复杂系统的管理和控制[J].  控制与决策, 2004, 19(5): 485-489.

WANG F Y. Parallel system methods for management and control of complex systems[J]. Control and Decision, 2004, 19(5): 485-489.

[11]  王飞跃.  软件定义的系统与知识自动化:  从牛顿到莫顿的平行 [J]. , 2015, 41(1): 1-8.

WANG F Y. Software-defined systems and knowledge automation: a parallel paradigm shift from Newton to Merton[J]. Acta Automatica Sinica, 2015, 41(1): 1-8.

[12]  王飞跃.  平行控制:  数据驱动的计算控制方法[J].  自动化学报, 2013, 39(4): 293-302.

WANG F Y. Parallel control: a method for data-driven and computa- tional control[J]. Acta Automatica Sinica, 2013, 39(4): 293-302.

[13]  王飞. 平行系统济系统计[J]. 杂性, 2004, 1(4): 25-35. WANG F Y. Artificial societies, computational experiments, and par- allel systems: a discussion on computational theory of complex so- cial-economic systems[J]. Complex Systems and Complexity Science, 2004, 1(4): 25-35.

[14]  白天翔,  王帅,  沈震,  .  平行机器人与平行无人系统:  框架、结构、[J]学报, 2017, 43(2): 161-175. BAI T X, WANG S, SHEN Z, et al. Parallel robotics and parallel unmanned systemsframework, structure, process, platform and ap- plications[J]. Acta Automatica Sinica, 2017, 43(2): 161-175.

[15]  郑南时代[J]. 能科学与, 2019, 1(1): 1-3.

ZHENG N N. The new era of artificial intelligence[J]. Chinese Journal of Intelligent Science and Technology, 2019, 1(1): 1-3.

[16]  王飞跃.  行矿山件定义的山到智能矿山[J].  复杂与智能化, 2018.

WANG F Y. Parallel mines: from software-defined mines to smart mines[J]. Complexity and Intelligence, 2018.

[17]  刘腾,  于会龙,  田滨,  .  智能车的智能指挥与控制:  基本方法与系统[J]. , 2018, 4(1): 22-31.

LIU T, YU H L, TIAN B, et al. Intelligent command and control sys- tems for intelligent vehicles: primary methods and systemic construc- tion[J]. Journal of Command and Control, 2018, 4(1): 22-31.

[18]  刘腾,  ,  ,  . 基于数字四胞胎的平行驾驶系统及应[J].智能, 2019, 1(1): 40-51.

LIU T, WANG X, XING Y, et al. Research on digital quadruplets in cyber-physical-social space-based parallel driving[J]. Chinese Journal of Intelligent Science and Technology, 2019, 1(1): 40-51.

[19]  张辉,  聂百胜,  许滕,  .  煤矿虚拟现实技术应用与发展[J].  煤矿安全, 2016, 47(10): 118-121.

ZHANG H, NIE B S, XU T, et al. Utilization and development of virtual reality in coal mines[J]. Safety in Coal Mines, 2016, 47(10): 118-121.

[20]  驾驶地矿用汽车径跟踪速度决[D].  : 北京, 2016.

ZHAO X. Research on path tracking and speed decision of unmanned underground mining vehicles[D]. Beijing: University of Science & Technology Beijing, 2016.

[21] GUO H, SONG L, LIU J, et al. Hazard-evaluation-oriented moving horizon parallel steering control for driver-automation collaboration during auto- mated driving[J]. IEEE/CAA Journal of Automatica Sinica, 2018.

[22]  GAO Y, SHEN Y, ZHANG W, et al Robust path tracking control for autonomous heavy vehicles[J]. SAE Technical Paper, 2018, 1: 1082.

[23]  高玉.  铰接式工程车操纵稳定性制与可视化模型开发[D].  北京, 2018.

GAO Y. Handling and stability control for articulated steer vehicles and virtual reality model development[D]. Beijing: University of Science & Technology Beijing, 2018.

[24]  王晓,  要婷婷,  韩双双,  .  平行车联网: ACP 的智能车 网联[J].  学报, 2018, 44(8): 1391-1404.

WANG X, YAO T T, HAN S S, et al. Parallel Internet of vehicles: the ACP-based networked management and control for intelligent vehi- cles[J]. Acta Automatica Sinica, 2018, 44(8): 1391-1404.


10.png

11.png


Parallel End-to-End Autonomous Mining: An IoT-Oriented Approach

 

AbstractThis paper proposes a new solution for end-to-end autonomous mining operations: Internet of Things (IoT) based parallel mining, consisting of the concept definition, the solution given and the concrete realization. The proposed parallel mining is inspired by the ACP (artificial societies (A) for modeling, computational experiments (C) for analysis, and parallel execution (P) for control) approach. The basic framework of parallel mining is given and its advantages are expounded. Then the solution of parallel mining is proposed, which is mainly composed of four parts: the management and control center for autonomous mining, the autonomous transportation platform of truck, the semi-autonomous mining / shovel platform, and the remote take-over platform. Key technologies of IoT based parallel mining are discussed in detail, namely, network communication, virtual parallel mining construction, mining environment perception over-the-horizon for the moving area and obstacle detection, collaborative decision-making, planning and control for unmanned mining equipment, parallel taking-over and remote control. Finally, the performance of IoT based parallel mining, including fusion perception, collaborative decision-making, planning and control, are evaluated. The realization of parallel mining can fundamentally improve the safety of personnel and equipment, reduce the cost of mining operation and increase the production rate. 

 

        Index Terms—Internet of things, ACP approach, parallel mines, management & control, decision-making, autonomous truck 

I.          Introduction

Today, the world has entered a new era of intellectualized development. Cyber, physical and social systems can be integrated by the Internet of Things (IoT) which enables the perception, computation and execution in an intelligent design paradigm [1]. Under such a background, the mining industry is facing many challenges, such as relatively low price of mineral products, deeper exploitation of resources, poor working environment, etc. [2,3]. In addition, the serious aging of employees, the shortage of skilled workers, the rising cost of manpower and the stricter requirements of safety and environmental protection have brought great challenges to the development of mining industry[4, 5].

As the driving force of the automobile industry reform, auto-driving technology has not only become the focus of many traditional automobile enterprises all over the world, but also one of the focuses of major Internet giants. However, due to limitations of policies, security, technology, ethics and many other factors, the commercial success of unmanned vehicles in the open scenario on public roads is facing many bottlenecks. Noticeable obstacles includes complex traffic signs, unpredictable behavior of road participants and unstable hardwares [6]. While general purpose autonomous driving suffers from these difficulties, specialized applications for mining sites could be realized relatively easily and provide novel solutions to above mentioned challenges in the mining industry. On the one hand, by connecting the vehicles to the Internet of Things, they can transmit all kinds of information to the central processing unit to realize the intelligent traffic management and vehicle control [7]. On the other hand, the special vehicles under limited scenarios with low or medium speed conditions will be the first rigid requirement for auto-driving technology to commercialize. As a relatively closed scenario, mining system has become one of the most important areas concerned by many auto-driving related enterprises [8,9]. As the mining and transportation operation in the mines implied a degree of repeatability and have some commonly used conditions, the construction machineries, such as excavators, wheel loaders, mining dump trucks, etc., first started the process of unmanned refit [10,11].

Unmanned mining has huge market potential, which can bring the mining enterprises (users) with cost, safety, efficiency and environmental benefits. Correspondingly, it will bring huge profits to the mining equipment manufacturers. It has attracted competing investment from mining equipment manufacturers, self-driving technology companies and large mining enterprises. However, the proportion of mines using automated equipments is still very low at present. With the pursuing for zero casualties and entering the era of skilled workers shortage in developed countries, it is estimated that by 2020, 50% of the major mining companies in Australia, Europe and the Americas will use the unmanned products, and some mines have decided to adopt unmanned mining equipments wholly. [12]

To realize the end-to-end autonomous mining operations, Komatsu launched the Autonomous Haulage System (AHS) to operate and manage fleets of self-driving mining trucks with capacities between 200 and 400 tons [13]. Caterpillar released Cat MineStar system to optimize productivity, enhance security, and improve machine utilization [14]. Sandvik developed autonomous loaders and trucks which operate underground [15]. Volvo tested it’s self-driving tipper inside the earth at a depth of 1,320 meters and even with artificial lighting [16].

Overall, it has been possible to realize the autonomous driving on a single mining equipment no matter in an above-ground mine or an underground mine. However, more researches are needed on the control robustness of unmanned equipment, on the cooperative operation among equipment and on the cluster operation to achieve the overall improvement of safety, cost and efficiency of unmanned mines. As a typical complex system, mine system includes production subsystem, dispatching subsystem, water and power supply subsystem, transportation subsystem, management and control subsystem, safety subsystem etc. Of which production, transportation, dispatching and safety are the key subsystems in realizing the unmanned operation of mining system [17]. Faced with the control and management of such a complex and huge system, parallel theory [18,19] and ACP approach [20-22] provide a good solution for mining intellectualization and unmanned operation. The parallel mines inspired by parallel theory and ACP approach is a highly networked intelligent system, which is in the scope of IoT. The IoT based parallel mining also refers to humans, excavators, off-road dump trucks, high-speed heavy trucks and other devices connected in the mining environment.

By using IoT technology, the real-time data of key systems in mining are collected and transmitted to the management center, where a dynamic artificial system corresponding to the real system are constructed. This artificial system is proposed to simulate the real complex system in real time, dynamically and on-line. By studying the evolution and prediction of artificial systems, the prediction and management of real mining systems are realized.

The main contributions of this work include: 1)The proposing of the framework and solution for IoT based parallel mining, which may improve the safety of personnel and equipment, reduce the cost of mining operation and increase production rate by improved efficiency. 2)The key technologies motivated by IoT thought [23-25] in the implementation of autonomous mining are introduced: from the network communication, model construction, integrated perception, to the collaborative decision-making, take-over and control.

The rest of the paper is organized as follows. Section II describes the framework of the IoT based parallel mining. Section III gives the solution for the realization of parallel mining. The key technologies of parallel mining are introduced in Section IV in detail, which mainly include: the network communication, virtual parallel mining construction, mining environment perception, collaborative decision-making, unmanned mining equipment control, as well as parallel taking-over and remote control. Section V evaluates the performance of the key technologies based on Internet of Things in parallel autonomous mining. Finally, our conclusion is provided in Section Ⅵ.

II.       The Framework of Iot Based Parallel Mines

The concept of parallel system was first put forward by F. Wang in 2004 [26], that is, to study the control, management, optimization and guidance of complex systems by constructing the virtual-real interactive artificial system and real system model. In parallel system, the ACP approach is the hard-core technical section, in which A refers to artificial society, C refers to computational experiments and P refers to parallel execution [27,28].

As a relatively closed and structured environment without interference from foreign vehicles and pedestrians except for operating equipment and professionals, mining system becomes one of the main scenarios in which driverless technology can be realized quickly. The driver's high labor intensity and poor working conditions also promoted this progress. Compared with the unmanned operation of single equipment, the cluster cooperative unmanned operation among all the mining equipments is more needed, which is exactly the important solution provided by parallel driving [27]. Parallel driving could obtain the data and scenarios in perceptual limits and other situations via the describing system. Knowledges and experiences can be shared to all vehicles in the system through analysis of a large number of algorithms and large-scale computation and evaluation of driving decisions. Relying on a single vehicle’s own ability of perception, decision-making and control, as well as the powerful GPU behind it for data processing, the vehicle itself possess the ability to adjust its own behavior according to the environmental changes. The human remote control is only activated when the early warning had occurred. The IoT based parallel mining is an intelligent solution that applies parallel driving technology to the mining scenarios.

The framework of IoT based parallel mining is shown in Fig.1. It is a true embodiment of parallel system in the field of mining. The parallel mines has five important locations, namely the loading site A, the dumping site B, the transfer belt unloading site C, the freight station D, the parallel management & control center E. Among them, A, B, C and D can be regarded as real systems, while E and F can be regarded as virtual systems.

Within the parallel mining system, we mainly study and solve four problems: the first is the semi-autonomous excavation at point A, which improves the efficiency of excavation through the cooperation between excavator and mining truck. The second is the autonomous transportation of off-road dump truck from point A to point B. By installing sensors and developing algorithms of intelligent sensing, decision-making and control, the dump truck could realize it’s autonomous driving. The third is the autonomous transportation of high-speed heavy truck from point C to point D. Unlike off-road dump truck, the high-speed heavy truck is operated on the open road and running in a relatively high speed. It requires higher level automated driving capacity which could avoid possible obstacles. The fourth is the management and control of parallel unmanned mining, which is responsible for the communication, supervision, coordinated control and dispatch for the automatic operation of all equipment. The end-to-end unmanned mining operation can be realized by the cooperation of these four sections, which aims at improving the safety and efficiency of the whole system and reducing the cost of mining operation.

                                               1.jpg

Fig.1 The framework of IoT based parallel mines

 

In the parallel mining, parallel subsystems corresponding to their real counterparts will be established to realize the physical system in real space mapping into virtual space. It could essentially solve the problems of low degree of network connection, difficult scheduling and confused management of each subsystem. Driven by data, a series of virtual mining organizations and institutions that interact with the actual mining system are constructed to form the parallel unmanned mining system. Various complex problems including mining scheduling, coordination, management, control and communication are constantly analyzed and evaluated by means of computational experiments. The aim is to change the production mode of the mining system and improve the synergy level and intelligence degree between the mining subsystems. The parallel mining divides a complex huge mining system which is difficult to manage, control and forecast into several parallel subsystems according to its functions, which can be flexibly optimized. At the same time, the forecasting and management strategy explored in the virtual world through computational experiments and parallel execution can be applied on the real mining system, enabling the real mining system’s ability of coping with unknown work scenarios and tasks.

III.     Solution for Iot Based Parallel Mines

The IoT based parallel mining system consists of four parts: the parallel management and control center (hereafter called M&C center), the autonomous transportation platform, the semi-autonomous mining/shovelling platform, and the remote take-over platform. As the nerve center of the mining system, the parallel M&C center is responsible for comprehensive real-time monitoring and comprehensive dispatching of unmanned mining equipment. The real-time interaction and two-way optimization through the established virtual mining system and their real counterparts ensure the safe and efficient operation of actual mining. The semi-autonomous mining/ shovelling platform and the autonomous transportation platform are equipped with a variety of sensing devices to realize semi-autonomous excavation, loading, unmanned transportation and unloading of minerals. The remote take-over platform is responsible for monitoring the operation of the mining equipment. Under the initiative request of the takeover from the semi-autonomous mining/shovelling platform or autonomous transportation platform, or in a special emergency, the remote take-over platform realizes the intervention and takeover for multiple devices according to the decision made by the parallel M&C center.

 

2.jpg

Fig.2 Solution for IoT based parallel mines

 

A.        The parallel M&C center

The parallel M&C center consists of a description system defined by the virtual parallel mining system, a prediction system defined by big data analysis and the deep learning center, and a guidance system defined by the dispatch center. Various data generated during mining production and operation are transmitted to the information processing center, where each subsystems parallel to the actual mining operation are constructed through data fusion, data mining, and visualization processing. In this way, on the one hand, the virtual data are built using actual data to complete parallel, online and real-time simulation of all aspects of mining production operations; on the other hand, the prediction, evaluation and optimization of real systems are completed through the evolution of parallel systems and collaborative interaction with real systems, thereby achieving the management and control of the unmanned mining system. The hardware in parallel M&C center includes simulation equipment, industrial computers, servers, video monitoring devices, image splicers, remote network equipment, etc. Each device corresponds to the function of the control center. In particular, if a dangerous situation occurs at the mining site which may affects the normal operation of other systems or the safety of workers, the control center will make intelligent decisions and monitor the mining site in real time.

B.        The autonomous transportation platform

The autonomous transportation platform mainly includes off-road dump truck and high-speed heavy truck, which are used for short and long distance transportation, respectively. The autonomous truck is equipped with lidar, radar, inertial navigation system, camera, V2X equipment, industrial computer, wireless network tester, on-board monitor, etc. The transportation route of autonomous truck is assigned by the parallel M&C center. The truck runs automatically at a suitable speed in the cycle of loading, transportation and unloading according to the target route, its own position and the surrounding environment.

The autonomous transportation platform includes functions such as autonomous driving, mode switching, status detection and display, information communication and management. The mining environment information perceived through lidar, radar, camera, as well as the positioning information provided by the inertial navigation system, are provided to the decision-making subsystem and control subsystem for the decision, planning and control. The communication subsystem is responsible for the wireless communication between the unmanned equipment, the M&C center and the remote take-over platform, thus providing support for the remote monitoring and control, and the cooperation between excavators and mining trucks. The data management subsystem performs data backup for the operation of autonomous truck, which can be used for data playback and further research. Considering the special requirements of mining operation safety, the autonomous transportation platform also has a state detection subsystem and a mode switching subsystem. The state detection subsystem displays the health status of each section. Integrating the data and commands from the state detection subsystem, the M&C center, and the remote take-over platform, the mode switching subsystem make decisions on the switching between manual mode and the automatic mode.

C.        The semi-autonomous mining/shovelling platform

The semi-autonomous mining/shovelling platform is equipped with sensors such as displacement sensor, pressure sensor, lidar, inertial navigation system, camera, radars, etc. The excavators or loaders cooperate with the autonomous truck by semi-autonomous mining or autonomous shoveling at mining sites.

In the process of the cooperation between excavators and mining trucks, the driving path for the mining truck in the working area is firstly planned according to the position of the excavator, and the excavator guides the mining truck to the correct position for loading. Then the excavator fulfill the automatic excavation of target minerals [29,30] and automatic unloading into the container of the mining truck according to the sensor information. During the entire excavation process, the human operator in the control center could monitor the excavation process and manually intervene when necessary, which not only ensures the smooth progress of the automatic excavation, but also greatly improves the working environment of the driver.

For the cooperation between the loader and the mining truck, after the mining truck is guided to the target position, since the loader moves more conveniently, the self-loader is responsible for the mineral loading, driving route planning and tracking, and finally depositing the minerals into the truck container.

D.       The remote take-over platform

To fulfill the high safety requirements for unmanned mining operation, the remote take-over platform is responsible for the safe manual control for the autonomous mining truck and the semi-autonomous excavators/loaders at emergency situations [31]. The remote take-over platform implements different processing strategies according to the fault levels, and designs a multi-level policy of security confirmation for mode switching and takeover. The remote take-over platform can realize the control for multiple autonomous mining trucks or excavators/loaders. It could also realize the cooperation between the excavator and the mining truck in the remote take-over state through the monitoring and dispatching of the M&C center. The remote take-over platform improves the operation quality and productivity while increasing the safety of human drivers and the equipment.

IV.     Key Technologies of Iot Based Parallel Mines

This section introduces the key technologies we studied in the implementation of IoT based parallel mining, which mainly includes: the network communication, virtual parallel mining construction, environment perception, the collaborative decision-making, unmanned equipment control, as well as parallel taking-over and remote control. The details are as follows.

A.        Mining network communication

Due to the influence of signal attenuation and Doppler effect caused by occlusion, air moisture content, equipment deployment, etc., the quality of wireless communication will be disturbed to a certain extent. It is necessary to design and choose a reasonable communication mode and equipment to ensure the security and reliability of information transmission. For traditional WiFi, the one-way transmission time of UDP packets may reach hundreds of milliseconds to seconds when the network is congested; For Mesh wireless network, the probability of packet loss and delay increases with the increase of hops, and the network topology will change and needs a certain time to converge, which further increases the probability of packet loss and delay.

To ensure the reliability and stability of network communication, we use a combined network scheme of WiFi-Mesh in parallel mining network communication. The principle of the network communication in parallel mining is shown in Fig.3, which is divided into the remote communication between equipment and center and the terminal management communication between different equipment. In the cooperative communication between mining truck and excavator, the Wifi-Mesh communication equipment is installed on roadside, enabling excavator and mining truck to realize the V2V communication. The CPE communication equipment is installed on excavators and mining trucks, and connected with the 4G base station to realize the V2Server communication.

In order to enhance the reliability and the anti-jamming ability of communication system, certain strategies, such as deploying multi-communication system, increasing link budget redundancy, increasing the budget margin of frequency band and bandwidth, are adopted in the WiFi link. The network transmission protocol can be selected according to the characteristics of various communication protocols, scenarios and applications.


4 (2).png

Fig.3  The principle of mine network communication

 

In the network communication, there will inevitably generate communication abnormalities, such as SUDP packet loss, the communication disconnection between vehicle and control center, etc. A well-designed exception handling mechanism can effectively reduce the impact of network anomalies. Abnormal handling methods include: displaying the loss of packets to the administrator, recording events and logs; alarming the control center in time to inform the administrator when communication disconnection occurs, or parking directly to ensure safety.

B.   Virtual parallel mining construction

As the foundation of parallel M&C center of unmanned mining, the construction of virtual parallel mining is essential. In practice, the mining system is complex and changeable, and it is difficult to build a reliable mathematical model. Therefore, the mine production process is difficult to be controlled accurately, and of great safety hazard. To overcome these difficulties, information technology is used to collect data of key dynamic attributes for each subsystem in mining system. Besides, digital modelling, virtual technology and visualization technology are used to establish artificial system corresponding to the system in reality, and the collected data are imported into the artificial system in real time, making the artificial system parallel to the real system, which can be called as parallel system.[32] Furthermore, in the parallel systems, various intelligent algorithms and optimization algorithms are used to predict and optimize the operation of the real system, and send the results to the real system, so as to realize the optimal control of the real system.

Taking the autonomous truck as an example, the structure of virtual parallel mining is shown in Fig. 4, and its visual display scene is shown in Fig. 5. The autonomous truck simulation platform in virtual parallel mining covers the module of vehicle dynamic model, virtual reality model, virtual sensor and environment perception, deviation calculation, planning and decision-making, control, etc. The output data include camera videos from each view point, radar data, 3D coordinates of vehicles, heading angle, speed, steering wheel angle, throttle/brake pedal’s position, tire forces and so on.

The input of the platform is the relevant data generated by the actual truck, including the current position, vehicle speed, the steering command from the controller, and the pedal instruction. After the collected data have been imported into the artificial system in real time, the autonomous truck in virtual parallel mining is synchronized with the real truck. With this platform, we can accurately simulate the truck performance and predict possible problems. It can also quickly verify the effectiveness of the autonomous driving algorithms in planning, decision-making and control, especially in various extreme conditions. Finally, the prediction, control and management strategies will be applied to the real mining system.

 

3.jpg

Fig.4  The structure of virtual parallel mine

 

5.png

Fig.5 Visual display of parallel mine

 

C.        Perception over-the-horizon in mining Environment

The environment perception system of each mining truck uses a collection of sensors, such as lidar, millimeter-wave radar, camera, ultrasonic wave sensor, infrared sensor, etc., to perceive the environmental information. During the operation of mining machines, multi-sensor fusion technology is used to monitor the driving area in order to detect and track the obstacles. At the same time, it can identify specific targets to avoid collision and provide warning information while reversing.

Multi-sensor data fusion is also used to eliminate the data redundancy between different sensors, and enhance the robustness of the system through data complementarity, so as to improve the reliability of the environmental sensing system. The sensors and their coverage areas of the mining truck are shown in Fig.6. The overlapped coverage areas of multiple sensors are the working areas of data sensing and fusion.

 

6.jpg

Fig.6  Sensors & it’s coverage of the mining truck

 

The resolution of sensors used in our truck is usually not very high, which reduces the capital cost, at the expense of moderate perception ability lose. In addition, the road width in the mining area could easily reach 30-40 meters, and the effective perception range of sensors in dust/sand environment is greatly reduced. Therefore, it is difficult to make safe and efficient decisions based on the perceptual results of a single vehicle in complex traffic scenarios. Thanks to the cluster management and control system and V2X communication technology, all perceptual results, such as vehicle localization, status information and surrounding environment perception information, can be uploaded to the cluster center. The center can further send the perception results of other vehicles, within a certain range around it, to a vehicle, so as to realize the sharing of perception results among multiple vehicles and achieve the ability to perceive over-the-horizon. The principle of fusion perception over-the-horizon is shown in the Fig.7.

 

7.png

Fig.7 Perception over-the-horizon by sharing

 

D.       Collaborative decision-making

Based on the perception results of over-the-horizon for mining environment acquired via V2X communication, the scheduling and management of the fleet in complex mining scenes can be realized by designing the decision-making algorithm for autonomous mining trucks. In the parallel autonomous mining transportation, the collaborative decision-making between the management & control center and autonomous truck is designed, which could greatly improves the efficiency of transportation for each autonomous truck. In order to ensure the reliability of the unmanned system, each autonomous truck still maintains its own decision-making within the range of its perception ability. In addition to emergency scenarios, the decision-making instructions issued by the center have the highest priority. If the center does not issue any instructions, the decision-making by the autonomous truck is used.

In areas beyond the perception range of a single vehicle, the decisions can be made by the M&C center, or by each autonomous truck according to the information from other vehicles that is issued by the center. The comparison of two decision-making modes is shown in Fig. 8. When a decision is made by the center, all vehicle parameters are calculated only once at the central terminal, and the processing speed of decision-making is faster because of the higher central computer specs. When a decision is made by each autonomous truck, the center needs to send the perception/location information of surrounding vehicles to each vehicle, which not only increases the amount of data transmission, but also increases the network communication burden. And the decision-making process is slower because of the limited computing power of the calculation unit mounted at each vehicle, and the multiple calculations carried out on each vehicle. For example, assuming that there are n autonomous truck in the whole mining area, all the positioning and sensing information of the truck is uploaded to the central terminal. The network occupancy rate is very low when the decision is made by the center, which does not need to send a large number of vehicle perception information to individual vehicles. Only one calculation was made at the central terminal, and then the decision-making instructions is sent to individual vehicles. However, if the decision is made by individual autonomous trucks, each vehicle needs to do a decision-making calculation (totally n times of calculation) in addition to sending a large number of perception data to surrounding vehicles. Moreover, if the decision is only made by autonomous trucks, the traffic problem among the autonomous trucks can not be effectively coordinated.

Therefore, the integrated dispatch of all vehicles through the fleet management system at the central terminal has the advantages of fast decision-making speed and low network burden, which can effectively improve the transportation efficiency of autonomous trucks and ensure the traffic safety.

After all the vehicle perception information is uploaded to the central terminal for decision-making, the fleet management of autonomous truck is transformed into the decision-making and planning for traffic flow. Here, the decision-making at the intersection is taken as an example to illustrate. The road in the mining area is specially designed. Traffic lights are usually not used at intersections, so the “first come first go” traffic rules are adopted. Under this traffic rule, vehicles need to slow down or stop at the intersection before passing, which affects the transportation efficiency of mining area and increases the fuel consumption of transportation. In parallel autonomous mining transportation, the M&C center predicts, plans, and manages all vehicles passing through the intersection in advance, automatically manages the order and speed of vehicles, avoids collision or parking at the intersection, and enables all vehicles to pass through the intersection in efficient and smooth motion.

 

8.png

Fig.8  Comparison of decision-making made by center and by vehicle

 

The principle of collaborative decision-making at intersection is shown in Fig. 9. Two circular areas are designed in the intersection area: the rescheduling area between outer cycle and inner cycle for passing order and speed planning, and the crossing area within inner cycle with determined rules. When a new vehicle enters the outer circle in a certain speed, the velocity and passing order of all vehicles within the annulus will be rescheduled. Once a vehicle enters the inner circle, its planned passing order and speed will not be changed for safety reasons. It should be noted that the decision-making algorithms should arrange the velocities of each autonomous truck to ensure that they will not collide in intersection area. [33,34]

E.        Unmanned mining equipment control

Mining equipment, such as off-road dump trucks, high-speed heavy trucks, excavators, loaders etc., are much different from conventional passenger vehicles. Their size and weight are very large, and the used transmission and steering systems also have certain particularities. Therefore, the mining equipment have many unique characteristics, such as large inertia, large system delay and large variation of load. In addition, the working environment of the mining site is harsh and the pavement conditions are complex. There are a lot of continuous climbing, turning and uneven pavement. Therefore, both the facility performance and working environment are more complex than conventional passenger vehicle’s scenario. It makes the control of these equipment more difficult, which requires the control algorithm to have better adaptability and robustness.

  9.png

Fig.9  Collaborative decision-making at intersection

 

For the lateral control in the course of driving, the steering system needs to be controlled by the path following control algorithm according to the information provided by the integrated navigation and positioning system and the planned ideal trajectory. The path-following control based on the virtual terrain field (VTF) method is used for the heavy mining truck control [35,36]. For the VTF control, a virtual U-shaped terrain field is assumed to exist along the reference path. The altitude of terrain field will goes higher as the lateral error becomes larger. If the vehicle deviates from the reference line, additional lateral restoring forces caused by the virtual banked road will be applied on the vehicle. The vehicle will be pulled back to the lowest position (reference centerline) under the influence of the additional lateral forces.

 

10.jpg

Fig.10 The processing flow of VTF control

 

Fig.10 is the processing flow of VTF control. First, the tracking errors is calculated according to the digital map and parameters acquired by sensors. Then the function of VTF altitude and the lateral tire forces are established and derived, respectively. According to the 2-DoF vehicle dynamics model, the target steering angle are derived, which is applied to the vehicle for path-following control.

F.        Parallel taking-over and remote control

The large-scale unmanned mining is a centralized operation business scenario, in which thousands of engineering machines (including mining truck, semi-autonomous excavator, patrol cars, etc.), surveillance cameras and various sensors, etc, are used. It should be pointed out that the unmanned mining scenario is not completely unmanned operation. Its main meaning is the unattended operation and intelligent decision-making for mining. Therefore, in the current planning, a certain number of professionals are still needed to manage and assist decision-making through appropriate takeover or remote control, so as to maintain the normal operation of various equipment.

In the remote supervision, the experienced drivers can send takeover request for the supervised mining trucks in the control center. Through the identification and processing of the central processor, the command can be sent to the specific trucks that need to be controlled, so as to realize the remote control by driving simulator. The main purpose of the remote control is to ensure the safety of the truck through human operation in the case of abnormal condition, so as to enhance the robustness of the whole intelligent mining system.

For parallel unmanned mining system, a large number of unmanned equipment and a certain amount of takeover drivers are need. The specific number of equipment needed to be supervised by each driver needs to be allocated reasonably. Insufficient supervisors can not guarantee the safe and reliable operation of equipment, and excessive supervisors will cause waste of personnel, costs and resources.

The decision-making process of remote takeover for parallel unmanned mining is shown in Fig.11. The detailed process is as follows:

Firstly, it is needed to judge whether the takeover is necessary according to the emergency situations such as the break down of the equipment or large path tracking error. If the remote takeover is needed, the take-over index of the equipment is calculated according to the parameters such as position deviation, signal fault, speed and acceleration. By comparing the demand index of all the equipment that need to be takeover, the priority grade of takeover of each equipment is determined. After the takeover index and the priority grade of all monitored equipment are determined, those equipment that need to be takeover can be controlled. For the equipment with the highest priority grade of takeover (pi=1), the monitor screen and control are directly allocated to the takeover driver for remote control; If other equipment request to takeover at the same time, it is necessary to determine whether the driver is sufficient. If the takeover grade of the equipment is less than or equal to the total number of takeover drivers, the monitor screen and control are allocated to one of the drivers. Otherwise, there is no remaining driver to takeover. At this time, braking/parking control must be carried out on the equipment with low level of takeover to ensure safety. When a driver completes the takeover for an equipment and is idle, the stopped equipment that cannot be takeover due to insufficient driver can continue to takeover until all the equipment are completed.

 

11.png

Fig.11 The decision-making process of remote takeover

 

In the way of remote takeover, due to the difference of the specific operation, the control mode of different equipment is also different. For driving-oriented equipment (such as mining truck), the control mode is relatively simple, and remote control can be carried out by driving simulator [31]. For excavator, loader or other equipment, excavation, shovelling and loading operations are also needed besides driving. This paper takes the remote gesture control of excavator as an example to illustrate:

The remote operator of excavator wears the wearable equipment on his upper body as shown in Fig.12, which can measure the three-axis angle of the arm, and then calculate the spatial position of the hand and some arm postures. The relevant control program is designed to realize the movement control of the excavator arm bucket in tracking the spatial position of the human hand and the action control of each mechanism of the excavator arm. The movement of the excavator includes the swing and lifting of the main boom, the lifting of the stick boom and the swing of the bucket. The left hand is responsible for switching gears, and the right hand is responsible for specific displacement control. The specific implementation scheme is shown in Fig.12.

The three kinds shift switching control of left-handed attitude are tracking control (A), stopping control (B) and separate mechanism control (C). When tracking control, the left hand sags naturally, which is used to track the spatial position of the human hand in real time by the excavator bucket; the left arm is placed in front of the chest to indicate that the excavator arm stops moving; and the mechanism is controlled separately to indicate that the four main movements of the excavator are controlled separately.

 

12.jpg

Fig.12 The remote gesture control of excavator

 

Thanks to the virtual parallel mining and equipment models which correspond to the real system, the video around the equipment is no longer need to be retransmitted during remote operation, but only the main parameters such as the position and attitude of the equipment are sent to the remote takeover terminal. Based on the limited data, the virtual equipment model is directly driven and displayed to the operator, which is used for the remote operation. As there is no need for video data transmission, the general 4G network can meet the network bandwidth requirements and achieve better control effect. Otherwise, it will cause a greater pressure on network communication, and 5G must be used to meet the demand.

V.       Performance Evaluation of Iot Based Parallel Mines

This section evaluates the performance of the key technologies based on Internet of Things in parallel autonomous mining, including the fusion perception, collaborative decision-making, planning and control.

A.        Perception

The detection module can collect information of road surface, driving area and obstacles, and provide reliable guarantee for safe and fast autonomous navigation. For road surface and driving area detection, lidar, camera and other sensors are used to extract roads and driving areas, so that mining machinery can reach the target point safely within the driving area when GPS positioning fails. As for target detection and tracking, by processing the sensor data from lidars and cameras, the obstacle information is detected and the dynamic obstacles are tracked. This information is transmitted to the planning module to realize the autonomous obstacle avoidance.

As the road for mining truck running is the unstructured type, it makes the road detection much difficult. First, the point cloud data of lidar is processed to eliminate the noise interference and to determine the grid map [37,38]. Then DBSCAN clustering is conducted to extract the ground point [39]. As the features of scanning points in vertical direction are distinct after dilation operation [40], if the difference between the maximum and minimum value of each grid is less than a set threshold, the grid is considered to be the ground, otherwise it is considered an obstacle. An example result of road pit detection is shown in Fig. 13 (a).

Ground points are fitted with Hough transform so as to extract the straight lines from laser scans on the ground and the end points of lines are taken as the contour points. Quadratic curve fitting with least square algorithm is used to find the road boundaries. The real detection results for road boundary and surface of the unstructured road are shown in Fig. 13 (b), (c).

The clustered obstacles, including pedestrians and vehicles, are fitted with boundaries. As the two-dimensional shape of the vehicle is rectangular, the L-shape fitting algorithm [41] is used. The minimum area rectangle (MAR) method [42] is used for pedestrian detection. The detection results are shown in Fig. 13 (d), (e).

 

13a.png

(a) road pit

 13bc.png

(b) road boundary                          (c) road surface

 13de.png

 (d) pedestrians                          (e) vehicles

Fig.13 Detection for road and obstacle

 

B.        Decision-making & planning

(1) Decision-making at intersection

To verify the effectiveness of collaborative decision-making for the fleet management of autonomous mining trucks, the whole time to pass through an intersection with different arriving rate λ (traffic load) are simulated. The results comparing with "ad hoc negotiation based" strategy are given in Fig.14. The testing arriving rate of autonomous truck is varying from 0.05veh/(lane•s) to 0.9veh/(lane•s). When the traffic arriving rate is very low (e.g. λ=0.05veh/(lane•s), the performance of the collaborative decision-making strategy is similar to that of "ad hoc negotiation based" strategy. This in mainly because the vehicles pass through the intersection do not need to stop when the traffic rate is low, so the passing time are basically the same. But when the traffic arriving rate goes higher, the time to pass through the intersection by the "ad hoc negotiation based" approach increases much faster than that of the collaborative decision-making strategy, which means the collaborative decision-making strategy has a notably advantage in leading to a quick and efficient passing performance. [43]

 

14.png

Fig. 14. The time to pass through an intersection with different traffic load [43]

 

(2) Local speed planning

Once the start and end points for the operation are determined, the fleet M&C center distributes the global path and speed planning results to each autonomous mining truck. However, it is inevitable to suffer discontinuous region of the planned global speed. When switching between different speed regions, the large speed deviation could result in sudden acceleration or braking of vehicles. For urgent acceleration, the target value can be gradually achieved through continuous acceleration without dangerous. However, the urgent braking must be accurately executed immediately for extra braking distance could bring about collision and other dangers. Therefore, the local speed planning for the braking buffer area must be determined before the discontinuous region of speed, and the braking begins when the vehicle enters the area.

To verify the effectiveness of the local speed planning algorithm, the local speed planning and control for mining truck on circular road was simulated on virtual simulation platform. The preliminary global speed planning result with discontinuity is shown by the grey curve in Fig.15. It can be seen that there is a sudden change in the planned speed value at the junction of straight line and turning, especially during the braking at 10s and 41.3s. If not replanned to decelerate in advance, the mining truck will rush into the turning area at a higher speed, which will affect the comfort and even safety of the truck. Under the established local speed planning algorithm, the mining truck can be accurately braked in advance before 10s and 41.3s, respectively. When the truck arrives at the turning area, it can exactly reach the preliminary planned velocity (about 20km/h).

The local speed planning takes into account the braking ability of the mining truck by controlling the deceleration according to the initial planned speed within the preview distance and the current speed. The braking related operations can be carried out timely and accurately. As the actual power of the truck is considered, the driving speed changes smoothly and the maximum deceleration can also be limited, which improves the stability, comfort and safety of the mining truck.

 

15.png

Fig.15. The local speed planning to solve discontinuities

 

C.        Control

In view of the particularity of the mining equipment and their working environment, if the control algorithm is directly verified on the real facilities, it may cause unnecessary risks, increase the waste of humans and costs. With the virtual simulation platform, the control algorithm for unmanned equipment can be quickly, safely and fully validated. Simulations of steering control by VTF approach at annular road, double-lane change road and snaking road were conducted to compare with PID control and preview feedback control (PFB) [44].

 

16.png

Fig. 16. Results comparison by different type controllers

 

To evaluate the driving performance of different controllers, the average lateral error and the average driver steering load under different scenarios are compared in Fig. 16. The results show that the average lateral errors controlled by VTF approach has significant advantages with the minimum value under the three scenarios compared with the other two scenarios. Compared with PID control and PFB control, the VTF approach could reduce the lateral error by 61%~88% and by 36%~74%, respectively. At the same time, the steering load of all the control algorithms are basically the same under all scenarios.

In addition, we compared the computer resource consumption during the process of the steering control to evaluate the real-time performance of different algorithms. Besides PID and VTF control, the linear quadratic regulator control (LQR) and model predictive control (MPC) are compared in Table 1. The memory utilization of different controllers are the same value. However, the CPU utilization are much different.

 


 

The LQR approach has the maximum CPU utilization (as high as 30.9%) when its maximum iteration is set to 150. The CPU utilization of MPC approach is 7.2% even though the prediction domain is only 10 steps. The CPU utilization of VTF approach is only 1.9%, which is even smaller than that of PID control.

From the above analysis, it can be concluded that the steering control based on VTF approach has a robustness in fitting different working conditions without increasing the steering load. At the same time, the real-time performance is also excellent.

VI.     Conclusion

Inspired by the ACP approach, this paper presented the IoT based parallel mining as a new solution for end-to-end autonomous mining operations, to improve the safety of personnel and equipment, reduce the cost of mining operation and increase production rate fundamentally. As the four main parts of parallel mining, the M&C center, the autonomous transportation platform of truck, the semi-autonomous mining/shovelling platform, as well as the remote take-over platform, were discussed in detail. The key technological advances achieved in the construction of parallel mining, including the network communication, virtual parallel mine construction, mining environment perception, collaborative decision-making, unmanned mining equipment control, as well as parallel taking-over and remote control, were presented and discussed as examples.

The case study for performance evaluation of IoT based parallel mining were evaluated. From the study results, it can be concluded that the IoT-oriented approach has obvious advantage in perception, decision-making, planning and control. IoT based parallel mining has excellent real-time performance for decision-making and control, and could improves the stability, efficiency and safety of the autonomous trucks.

Our on-going work is devoted to the improvement and implementation of intelligent scheduling algorithm. Future work involves the statistical analysis of operation data of several unmanned mines, to evaluate the advantages of IoT based parallel mining solutions in reducing operation costs, improving efficiency and enhancing safety.

References


  1. M. A. Razzaque, M. Milojevic-Jevric, A. Palade, and S. Clarke, “Middleware for internet of things: A survey,” IEEE Internet of Things Journal, vol. 3, no. 1, pp. 70–95, 2016.

  2. Paul J. A. Lever and Fei-Yue Wang, “Intelligent Excavator Control System for Lunar Mining System”, Journal of Aerospace Engineering, 1995, Vol. 8, No. 1, pp. 16-24.

  3. Paul Lever, Fei-Yue Wang, “Intelligent Excavation Control for a Lunar Mining System”, Proc. of 4th International Conference on Robotics for Challenging Environments, Albuquerque, NM, 1994, pp. 97-105.

  4. Levelling the Playing Field: The Mining Association of Canada. Report. April,2015.

  5. Mining Feature: Fatigue Management for Haul Truck Drivers: A Low-Tech Solution for a Small Mine". Office of Mine Safety and Health Research, National Institute for Occupational Safety and Health. 28 October 2014.

  6. Nunes, Ashley, Bryan Reimer, and Joseph F. Coughlin. "People must retain control of autonomous vehicles." Nature 556 (2018): 169-171.

  7. Contreras J, Zeadally S, Guerrero-Ibanez J A. "Internet of vehicles: Architecture, protocols, and security." IEEE Internet of Things Journal (2017).

  8. "Map of unmanned enterprises in China, 2018 Edition". Zhigu news. Available: http://zhigu.news.cn/2018-09/29/c_129963390.htm.

  9. "2017 RBR50 List Names the Leading Robotics Companies of the Year ". Robotics Business Review. Retrieved 2017. Available: https://www. roboticsbusinessreview.com/companies/?companyType=rbr_50.

  10. Paul J.A.Lever, Fei-Yue Wang, D.Q. Chen, “A Fuzzy Control System for an Automated Mining Excavator”, Proceedings of 1994 IEEE International Conference on Robotics and Automation, San Diego, CA, May. 8-13, 1994, Vol. 4, pp. 3284-3289.

  11. Lever, Paul JA, et al. "Autonomous robotic mining excavation using fuzzy logic and neural networks." Journal of Intelligent & Fuzzy Systems 3.1 (1995): 31-42.

  12. X. Chen, M. He. "Framework system and key technology of intelligent mine construction." Coal Science and Technology, 2018, 46(2).

  13. Parreira, Juliana, and John Meech. "Autonomous haulage systems–justification and opportunity." International Conference on Autonomous and Intelligent Systems. Springer, Berlin, Heidelberg, 2011.

  14. Mining technology - CAT® minestar. Available: https://www.cat.com/ en_US/by-industry/mining/mining-solutions/technology-new.html

  15. Glass labyrinth: 38-tonne automated loader. Available: https://www.rocktechnology.sandvik/en/

  16. Volvo tests a self-driving truck in a swedish mine, Blows Our Minds. Available: https://www.guidingtech.com/61345/volvo-tests-self-driving -truck-swedish-mine/

  17. Min Zuo. The "Application of Parallel System Theory in Modern Mine Managemnet. " METAL MINE, 2011, 40(9):131-133.

  18. Wang F Y. "Parallel system methods for management and control of complex systems. " Control and Decision, 2004, 19(5): 485-489.

  19. Bai T., Wang S., Shen Z., et al., "Parallel Robotics and Parallel Unmanned Systems: Framework, Structure, Process, Platform and Applications. " Acta Automatica Sinica, 2017, 43(2): 161-175.

  20. Wang F Y. "Software-defined Systems and Knowledge Automation: a Parallel Paradigm Shift from Newton to Merton." Acta Automatica Sinica, 2015, 41(1): 1-8.

  21. Wang F Y. "Parallel Control: A Method for Data-Driven and Compu- tational Control." Acta Automatica Sinica, 2013, 39(4): 293-302.

  22. Wang F Y. "Artificial Societies, Computational Expermients, and Parallel Systems:A Discussion on Computational Theory of Complex Social- Economic Systems." Complex Systems and Complexity Science, 2004, 1(4): 25-35.

  23. Fortino, Giancarlo, et al. "Towards a development methodology for smart object-oriented IoT systems: A metamodel approach." 2015 IEEE international conference on systems, man, and cybernetics. IEEE, 2015.

  24. Fortino, Giancarlo, et al. "Agent-oriented cooperative smart objects: From IoT system design to implementation." IEEE Transactions on Systems, Man, and Cybernetics: Systems48.11 (2017): 1939-1956.

  25. Chen, Min, et al. "A Dynamic Service Migration Mechanism in Edge Cognitive Computing." ACM Transactions on Internet Technology (TOIT) 19.2 (2019): 30.

  26. Wang F Y. "Computational Theory and Method on Complex System." China Basic Science, 2004, 6(5): 3-10.

  27. Wang F Y, ZHENG N N, CAO D P, et al. "Parallel driving in CPSS: a unified approach for transport automation and vehicle intelligence." IEEE/CAA Journal of Automatica Sinica, 2017, 4(4): 577-587.

  28. Wang F Y. "The emergence of intelligent enterprises: from CPS to CPSS". IEEE Intelligent Systems, 2010, 25(4): 85-88.

  29. Paul Lever, Fei-Yue Wang and X. Shi, “An Intelligent Task Control System for Dynamic Mining Environments”, AIME Transactions on Mining, Metallurgy and Exploration, 1994, Vol. 174, No. 1, pp. 165-174.

  30. P. J. A. Lever, Fei-Yue Wang, X. Shi, D. Chen, “A fuzzy-behavior-based approach for controlling mining excavator bucket/rock interactions”, Proceedings of 1994 IEEE Industry Applications Society Annual Meeting, Denver, CO, USA, Oct. 2-6, 1994, Vol. 3, pp. 2126-2133.

  31. LIU T., YU H., TIAN B., et al. "Intelligent command and control systems for intelligent vehicle: primary methods and systemic construction." Journal of Command and Control, 2018, 4(1): 22-31.

  32. Li, Li, et al. "Parallel testing of vehicle intelligence via virtual-real interaction." Sci. Robot 4 (2019).

  33. Huile Xu, Yi Zhang, Li Li, Weixia Li, "Cooperative driving at unsignalized intersections using tree search," https://arxiv.org/abs/1902.01024

  34. Li Li, Fei-Yue Wang, "Cooperative driving at blind crossings using intervehicle communication," IEEE Transactions on Vehicular Technology, vol. 55, no. 6, pp. 1712-1724, 2006.

  35. Gao, Y., Shen, Y., Zhang, W., Wang, H. et al., “Robust Path Tracking Control for Autonomous Heavy Vehicles,” SAE Technical Paper 2018-01-1082, 2018, doi:10.4271/2018-01-1082.

  36. Gao, Y., Shen, Y., Cao, D., "Path-Following Control by Dynamic Virtual Terrain Field for Articulated Steer Vehicles" Vehicle System Dynamics, https://doi.org/10.1080/00423114.2019.1648837.

  37. M. Darms, C. R. Baker, P. E. Rybski, and C. Urmson, “Vehicle detection and tracking for the Urban Challenge Vehicle Detection and Trackin g for the Urban Challenge,” vol. 10,no. 3, pp. 57 - 67, 2008.

  38. Yang F., Zhu Z. , Dong X., et al. "Real-time dynamic obstacle detection and tracking using 3D Lidar." Journal of Zhejiang University (Engineering Science), 2012, 46(9):1565 - 1571.

  39. Ester, Martin, et al. "A density-based algorithm for discovering clusters in large spatial databases with noise." Kdd. Vol. 96. No. 34. 1996.

  40. S . Thrun , M . Montemerlo, and A. Aron, "Probabilistic Terrain Analysis For High-Speed Desert Driving." Robotics: Science and Systems. 2006.

  41. Y. Ye, L. Fu, and B. Li, “Object Detection and Tracking Using Multi - layer Laser for Autonomous Urban Driving,” ITSC, 2016 IEEE 19th International Conference on. IEEE, 2016.

  42. Matti Lehtomäki, et al. "Object Classification and Recognition From Mobile Laser Scanning Point Clouds in a Road Environment." IEEE Transactions on Geoscience & Remote Sensing 54.2(2016):1226-1239.

  43. Yue Meng, Li Li, Fei-Yue Wang, Keqiang Li, Zhiheng Li, "Analysis of cooperative driving strategies for nonsignalized intersections," IEEE Transactions on Vehicular Technology, vol. 67, no. 4, pp. 2900-2911, 2018.

Kritayakirana, Krisada, and J. Christian Gerdes. Autonomous vehicle control at the limits of handling. International Journal of Vehicle Autonomous Systems 10.4 (2012): 271-296.


 



https://m.sciencenet.cn/blog-2374-1211246.html

上一篇:[转载]IEEE/CAA Journal of Automatica Sinica被SCI收录
下一篇:[转载]【CAA新闻】2019平行智能大会在北京成功举办

3 郑永军 范振英 周忠浩

该博文允许注册用户评论 请点击登录 评论 (0 个评论)

数据加载中...
扫一扫,分享此博文

Archiver|手机版|科学网 ( 京ICP备07017567号-12 )

GMT+8, 2024-3-28 16:15

Powered by ScienceNet.cn

Copyright © 2007- 中国科学报社

返回顶部