In oil eld wells, while drilling for several kilometers below surface, high temperature damages the drilling tools. This costs money and time for tripping operations to change the damaged tool. Existing temperature mitigation techniques have several drawbacks including a long response time, analogue signal issues and human intervention. In this work, we empower the down-hole tools with a coordination mechanism to mitigate high temperature in soft real time by controlling a down-hole actuator through a voting process. The tools are represented by agents that control the sensors and actuators embedded in these tools. To implement the proposed system properly, a model of the drilling domain is constructed with all drilling mechanics and parameters, along with the well trajectory and temperature equations taken into consideration. The proposed model is implemented and tested using AgentOil, a multi-agent-based simulation tool, and the results are evaluated. Furthermore, the requirements of a real-time temperature mitigation system for Oil&Gas drilling operations are identi ed and the constraints of such systems are analyzed.
a composition of several drilling tools with various functionalities, searching for natural resources. Once found, they will be produced and re ned for public use.
While drilling, temperature increases with depth in a speci c rate. High temperature damages the tools, which costs time for tripping operations to change the damaged tool apart from the cost of the damaged tool itself. When these tripping operations are not planned, the time spent is called Non-Productive Time (NPT). With the intention to monitor the status of tools, sensors are attached to them in order to read temperature, in which the use of Cyber Physical Systems (CPS) concepts is important. Moreover, investing in technologies to manage and mitigate high temperature showed to be cheaper than investing in a technology to withstand high temperature.
In most of the existing systems, temperature mitigation is executed by the eld engineer up-hole (several kilometers above the down-hole tools) after data about high temperature is sent from the down-hole tools. This mitigation relies on mud cooler to cool the drilling mud, the latter will be in contact with downhole tools. However, this process su ers from the following three aws:
First, existing wells rely on analogue telemetry to undertake the communication between tools down-hole and eld engineer up-hole. However, analogue telemetry has the following couple of drawbacks: (i) Long response time: the time for data to be sent up-hole, and for the impact of the cooled mud to reach the tools down-hole. In deep wells (more than 1km), analogue telemetry bandwidth will be low (less than 1bit=second), as mud pulse speed is 1kilometer=second [27]. (ii) Signal issues: the analogue signal is unreliable in extreme drilling conditions like high temperature, as it maybe become noisy and sometimes even lost. Second, the e ectiveness of the up-hole actuator (mud cooler) in high temperature conditions may become ine cient (even in maximum power). Third, in existing operations, the monitoring, problem detection, and decision making are undertaken by humans. This makes the process error-prone, unresponsive, and fault-tolerant.
To overcome these aws, we propose a Multi-Agent System (MAS) empowering the down-hole tools with a coordination mechanism to mitigate high temperature autonomously in soft real time by controlling a down-hole actuator through a voting process. The system allows to convert the drilling process into an automatic CPS using MAS. In this system, the agents represent the downhole drilling tools, and each agent is aware of the temperature speci cations of its tool and it controls the sensors and actuators embedded in its tool. Moreover, the agents react to data read by sensors attached to them, by triggering a voting cycle to send commands a down-hole actuator (bit controller) aiming at mitigating high temperature. Thus, the proposed system allows down-hole tools to handle the situation and take actions without the intervention of up-hole entities when the eld agent is not aware of the situation down-hole. Drilling mechanics along with the well trajectory and temperature equations needed in drilling operations are modeled in the system as the environment where the agents interact. A tool agent votes to start the actuator in a speci c power level as per its temperature speci cations and the current temperature, and an overall single decision is aggregated as per the votes of all tools. We have implemented several known voting rules to aggregate the votes.
One of the challenges in oil eld automated applications according to [
This work is organized as follows: Section 2 investigates the state-of-the-art about the MAS and CPS applications in Oil&Gas industry and about voting systems with a short background knowledge about the drilling tools and technologies. Section 3 discusses the proposed system. Section 4 evaluates the proposed system and discusses the results. Section 5 presents a real-time analysis for drilling agents. Section 6 draws conclusions and states the future work. 2
Since this work proposes a system for controlling drilling tools, Section 2.1 introduces a short background knowledge about the drilling tools and technologies. Next, Section 2.2 discusses the literature of CPS and MAS in the Oil&Gas industry. Later, Social choice theory, and particularly voting systems as coordination mechanisms, are studied in Section 2.3. Finally, a discussion about the real-time compliance for MAS and CPS is provided in Section 2.4. 2.1
The Bore-hole Assembly (BHA) is a set of drilling tools with embedded electronics used to drill the well (or bore-hole). Following are the main components from bottom to top of the BHA (Figure 1): { Bit: It is used to crush and cut rocks, hence drill; { Rotary Steerable System (RSS): It is designed to direct the drilling process with continuous Rotation per Minute (RPM) of the bit. In addition, it reacts to the steerable conditions and adjusts the position of the BHA; { Logging While Drilling (LWD): It measures in real time -while drillingthe formation characteristics. Formation & evaluation sensors are embedded in these tools that measure data and give logs related to the drilled formation; { Measurements While Drilling (MWD): It embeds steering & direction sensors that read data to determine the position and direction of the tool compared to the Earth magnetic and gravity elds. Additionally, it includes the modulator responsible for transmitting collected data from all tools in the BHA. Data transmission methods may vary from drilling company to another, but the main method involves transmitting data to the surface as pressure pulses in the mud system (analogue signal) which may witnesses signi cant delay to reach the surface up-hole. Moreover, successful data decoding up-hole is highly dependent on the signal-to-noise ratio.
All tools are powered by two sources: 1. From the MWD, as it includes a
turbine that generates power from the ow of drilling mud. 2. From a battery
in the tool itself, when the ow of mud is stopped to add a new drilling tool
to the BHA. All tools have repair & maintenance embedded sensors in them,
which give the status of the tool and measure the temperature that our system
is mitigating. Finally, all tools have electronic boards that enable programming
them up-hole as per the needed requirements of measurements, and hence, the
proposed model can be programmed in these boards.
The role of CPS in the industrial domain has been discussed thoroughly in the
literature [
Other research works discussed how, by utilizing advanced information
analytics, networked machines will be able to perform more e ciently,
collaboratively and resiliently in soft real time. They speci cally proposed a uni ed
5-level architecture as a guideline for the implementation of CPS [
Most of the work in the MAS domain in Oil&Gas industry is still theoretical
and conceptual. However, there are rare concrete applications. For instance,
in [
In the domain of MAS, voting systems are an active area of research that enables
decentralized decisions [
Voting rules are de ned to handle the voting process considering some properties like Anonymity (votes do not disclose the voters), Neutrality (each candidate is treated the same) [21]. Voting rules vary in types and processes, hence in what follows, we present the most used rules in the MAS literature: 1. Plurality: Voters cast a single vote, and the candidate with most votes wins. 2. Borda count: Each voter ranks the candidates in order as per her preference. This ordering contributes points to each candidate based on the position that she is ranked by the voter. In other words, if there are m candidates, it contributes m 1 points to the candidate ranked rst, m 2 points to the second, and so on. Accordingly, the winner is the one with the highest points [24]. 3. Single Transferable Vote (STV): This rule requires up to m 1 rounds.
In each round, the candidate with the lowest plurality score is eliminated
and votes for that candidate will be transferred to the remaining candidates
in the next round [
Real-time compliance has been identi ed as a key feature for MAS [
In a later contribution [
This section presented the related work in the domains of MAS and CPS as an application in Oil&Gas industry, and it discussed the real-time compliance for MAS and CPS. Additionally, a survey of the most used voting systems in such applications are presented. Next section will discuss the proposed system. 3
This work proposes that tools down-hole mitigate high temperature autonomously in soft real time with a decentralized collective decision. Notably, it takes advantage of the current infrastructure allowing for communication between these tools, as they all send measured/logged data to the MWD in order to send them up-hole. Additionally, there is one actuator down-hole in the RSS, which is the bit controller. It is responsible for controlling the bit rotation, that a ects the speed. Reducing the speed means delaying the time to reach higher temperature. Although this leads to slower drilling, it mitigates the temperature raise, thereby save the tools from failure allowing them to drill deeper. Section 3.1 analyzes the drilling mechanics and parameters. Section 3.2 discuses the multi-agent and voting architecture of the system. In addition, the proposed voting system is presented and explained. 3.1
In the literature of Oil&Gas domain, there is no concrete model of the drilling process and operations. Such model should include the drilling and temperature equations needed to implement and evaluate the system properly. In this section, concepts from the drilling terminology discussed in our model are introduced.
Measured Depth (MD) is the length of the well-bore, while True Vertical Depth (TVD) is the vertical distance from the surface until the bit. TVD is particularly important in determining the down-hole temperature. Measured with accelerometers in MWD, inclination is the deviation from vertical, irrespective of compass direction, expressed in degrees. It is relating MD with TVD using the Pythagorean equation. TVD calculation equation using average angle method is shown in Equation 1.
T V D =
M D
Where: M D = measured depth between two readings in di erent depths; I1 = inclination at upper reading; I2 = inclination at lower reading. Assuming the Azimuth direction is not changing, T V Dnew will be calculated as: T V Dnew = T V Dold + T V D.
The rate of increase in temperature per unit depth in Earth is called
Temperature Gradient, a.k.a. Geothermal Gradient. Although it is location dependent,
it averages at 30 C /kilometer [15 F /kilofeet] [
DownHoleT emp = Surf aceT emp + T V D GeothermalGradient (2)
Set by the eld engineer, drilling parameters are used to control the drilling process. Mainly, we focus in our model on three essential parameters: { Force: represented by Weight-on-Bit (WOB) applied to the drill-string, and it is only controlled up-hole. { Rotation: represented by Revolutions-per-Minute (RPM) of the drill-string, and it is controlled up-hole but can also be altered down-hole in our system. { Flow rate: which is the mud aw rate circulating inside the drill-string to cool down the tools and carry cuttings resulted from the drilling process. Other conditions a ecting the drilling process are bit type (roller cones, diamond, etc.), bit characteristics (cutting structure, dullness on drilling rate, rate of tooth wear and bearing life, etc.) and the size of the hole. However, they do not change through out the drilling run, hence we did not consider them in our model.
The speed of drilling process or the Rate of Penetration (ROP) as a function of several parameters is shown in (Equation 3) [19].
W OB
ROP = K ap r (3)
Where K: Formation factor, is a constant related to formation hardness; W OB: function of WOB; r: function of RPM; aP : function of ow rate and bit characteristics (for detailed information, c.f. [19]).
Based on the BHA components (Section 2.1) in existing oil eld wells, we propose in our model the use of MAS in Oil&Gas drilling operations (Figure 2), with the intention to convert it to a CPS application. Consequently, this will include using intelligent agents instead of humans.
In Figure 2 down-hole, we represent each programmable down-hole tool in the BHA (MWD, LWD and RSS) with an agent. This agent is responsible for the sensors and actuators embedded in the tool. Each tool has di erent sensors that are used to measure three di erent kinds of data: Steering & direction data, repair & maintenance data, and formation evaluation data. Only the MWD agent controls the modulator which is responsible for the communication with eld engineer up-hole. Likewise, only the RSS agent controls the bit controller that changes the rotation of bit, which is the actuator concerned by the voting down-hole.
In Figure 2 up-hole, the eld agent is replacing the eld engineer. This agent is responsible for controlling the drilling process by changing drilling parameters when needed to drill ahead and reach the total depth. Additionally, it controls the up-hole actuator (mud cooler) that is responsible for cooling the mud, which will be in contact with all tools.
Even though controlling the rotation speed is not implemented yet down-hole, we argue that, the infrastructure to do so is present in the RSS. The bene t of decreasing the rotation of bit is to slow down drilling and delay reaching a high temperature zone.
All voting rules that use several rounds to determine the winner are not suitable in this real-time application since they take time to conclude a result. Therefore, we implemented in our system those that use only one round. However, these voting rules have the drawback of not supporting the Condorcet consistency criterion. Therefore, we also included the implementation of the Condorcet rule.
The decision to activate the bit controller as well as to choose the needed power level is based on the achieved e ciency in mitigating high temperature with a lower cost, i.e. if the bit controller power level 40% is enough, then there is no need to choose the 100% power level. The current temperature of the tool is read by a sensor attached to the tool. It will be compared with the temperature speci cation level, and if it reaches this level, the tool reacts by initiating a voting cycle to start the bit controller.
The vote represents the desired bit controller power level which is determined as per the temperature speci cation levels of each tool. The vote has to be from a discrete set of candidates of bit controller power levels, which is f0, 20, 40, 60, 80, 100g. 4
AgentOil [
We have used a laptop with the following features to conduct experiments with the simulation: Processor: Intel Core i5-2520M; CPU: 2.50GHz; RAM: 8GB; OS: Windows 7 Ultimate 64-bit. The simulation runs for one drilling run, and all parameters have initial values that the user can change if needed before the simulation starts. Once the simulation starts, all tool agents and actuators (down-hole and up-hole) are created and set accordingly in the environment. We set a simulation tick to be corresponding to one minute. Thus, the results will be normalized as speed is given in meter=minute.
Every tick, drilling mechanics and parameters are updated to calculate the ROP (Equation 3). The agents measure M D, and compute their T V D from it (Equation 1). Then, they calculate the current temperature as per the TVD, geothermal gradient, and surface temperature (Equation 2). Once they have their temperatures, the decision model in each tool is considered, and requests for starting actuators are sent accordingly to mitigate high temperature either to start the mud cooler up-hole when reaching the danger level of the tool, or to start the bit controller when reaching the critical level.
A simulation run can end either successfully (reaching total depth) or unsuccessfully (with a tool failure). On the one hand, if both mitigation measures up-hole and down-hole were insu cient, a tool fails and a tripping operation is needed to change the BHA. This means NPT for the whole system. Consequently, the simulation ends with a message saying which tool failed and with what temperature along with the corresponding depth. On the other hand, if no failure happens, and the RSS reaches the total depth, the simulation ends and a message is displayed explaining how much time it took to drill the well. Each experiment has been conducted tens of times and an average has been concluded. In each experiment, we have used a xed number of agents to normalize the results, as follows: Up-hole: one eld agent and one up-hole actuator agent; Down-hole: one MWD agent, three LWD agents, one RSS agent and one down-hole actuator agent. 4.2
Figure 3 shows the results in terms of time and depth of running the simulation with the proposed system (colored curves) and without it (gray curve). We have averaged the results for di erent values of xed WOB (the x-axis). Figure 3 (left) illustrates the time elapsed before failure (in other words, how much time the tools survived in high temperature environment). Correspondingly, Figure 3 (right) plots the depth before failure (in other words, how much depth was drilled before failure), which is the main goal of drilling a well. In both gures, three voting rules were examined: Condorcet (in green), Plurality (in red), Borda count (in blue).
From Figure 3 (left), it can be noticed that all colored curves are above the gray curve, which means that all runs of the proposed system (with di erent voting rules) are better than simulation runs without enabling the proposed system. This result should be taken relatively, as more time means a longer delay before reaching the goal, and hence more money spent. Therefore, it should not be considered alone without the impact on depth before failure. Figure 3 (right) shows that the colored curves are below the gray curve (signi cantly with high WOB). For example the green curve (Condorcet rule) has drilled with 30k lbf WOB till approximately 4356 m which is 16 m more than what the gray curve (without enabling the proposed system) has drilled. Even though this di erence in depth drilled may seem insigni cant, it is vital in extreme drilling conditions because drilling 16 m at a deep depth takes hours in such conditions, and each hour a large amount of money is being spent to operate the drilling rig and pay for various services. In this experiment, we investigate the behavior of the proposed system in simulation ticks to analyze the impact in mitigating high temperature in soft real time. All runs are done with xed WOB: 10k lbf. In this experiment, a chart of temperature of one of the tools throughout the simulation is presented. In which, the x-axis represents time (simulation ticks), and the left y-axis represents temperature in degC.
As seen from Figure 4, for this drilling run, the starting position was 3000 m, and the temperature corresponding for this depth was 99 degC. This explains the beginning of the left y-axis. The blue curve represents the actual temperature mitigated by the system, while the red curve represents the temperature if no mitigation is done, hence, the temperature increases linearly in time while drilling with xed WOB. We can easily notice that, compared with the red curve, the blue curve witnesses considerable drops throughout the simulation run. 5
The aim of the CPS proposed in this article is to make a step forward towards
a real-time temperature mitigation system for Oil&Gas drilling applications.
However, as been shown in recent studies in the literature, real-time compliance
is one of the most di cult challenges confronting the application of MAS in CPS.
In particular, Calvaresi et al. [
In time-critical environments such as Oil&Gas drilling operations, real-time compliance is a primordial element. However, since this work has been implemented in Repast, it cannot overcome the limitations imposed by this multiagent simulation environment. That being said, within the aforementioned limitations, this works tries to address the real-time compliance by relying on a voting system with a single-round voting rule. This choice is explained as follows. First, typically, decentralized decision making problems can be solved with techniques like Distributed Constraint Satisfaction Problems (DisCSP) [28], these solutions are time-consuming and cannot comply to a time-critical application such as Oil&Gas drilling. Second, voting systems are very e cient when it comes to small preference aggregation of choosing between a discrete set of candidates.
Yet, this paper presents a work-in-progress. Other real-time compliance aspects will be addressed such as requirement to initiate a voting, the voting time window, and the delay before the actuators are activated will be studied and addressed. Furthermore, the communication delay is another considerable factor that should be taken into account in order to enhance the real-time compliance.
Existing technologies in materials engineering are currently being tested for drilling in the temperature of 200 degC. Several materials are used for these tests from plastic-encapsulated electronics on a plastic board, to ceramic-encapsulated electronics on a plastic board, to ceramic and metal components (no plastics) [23]. Yet, beyond this temperature, there is a need for a mitigation process to handle the temperature instead of a material to withstand the temperature.
Figure 5 illustrates an example of the requirements of the real-time system we plan to build. In this example, there are 4 agents: one MWD, two LWDs, one RSS, and one actuator (bit controller). The temperature that damages the tools is 200 degC, and the RSS reaches the critical temperature of 195 degC. Although any tool can reach the critical temperature, there are two worst case scenarios when either the MWD or the RSS faces high temperature, as the communication time between down-hole tools in these cases will be maximized to send the requests to other tools, due the fact that the tools are connected in sequence. Therefore, in this example, we chose one of these two worst case scenarios. The time window in which the system should react to mitigate the temperature is T imeW indow = t(DamageT emp) t(CriticalT emp) = 5 to 10 minutes where t(x) represents the time when the tool reaches the temperature x. While drilling, high temperature damages the down-hole tools, and the existing mitigation process is insu cient, due to the physical distance and the fact that the communication is analogue between the down-hole tools and the eld engineer up-hole that controls the existing mitigation process. Most of the exciting works proposing to use MAS in Oil&Gas are conceptual, with no concrete application of MAS in the engineering aspect.
In the proposed system, The tools agents react to high temperature and socialize to mitigate this high temperature autonomously down-hole and in soft real time. Voting rules have di erent pros and cons, so we have implemented several voting rules in our voting system (Plurality, Borda count and Condorcet). LWD 1 LWD 2 RSS B it Legend R4 S4
V3 R3 S3
V2 R2 S2 V1
DT
A1
A2 Voting Time
Decision Apply Action Time (start actuator)
Time for the effect to take place
Time Tool Agent
Process
Actuator The results of the performed experiments show that the system mitigates high temperature by delaying the damage and allowing the tools to drill deeper. Finally, we discussed the requirements of a real-time temperature mitigation system for Oil&Gas drilling operations, and we have analyzed the constraints of such system.
Starting the actuator slows down the drilling, which means more time to reach the target, i.e. higher cost for the whole drilling process. On the other hand, not starting the actuator in case of high temperature damages the tools, i.e. NPT due to tripping operations to change the BHA. Therefore, there will be a need to update the tool agent decision model with a utility function to accommodate the trade-o .
Acknowledgements: This work is partially supported by the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation.
This work is partially supported by the Region Bourgogne Franche-Comte (RBFC, France) project UrbanFly 20174-06234/06242. 19. Osgouei, R.E., Ozbayoglu, M.: Rate of penetration estimation model for directional and horizontal wells. In: 16th International Petroleum and Natural Gas Congress and Exhibition of Turkey (2007) 20. Rosenschein, J., Procaccia, A.: Voting in cooperative information agent scenarios: use and abuse. Cooperative Information Agents X pp. 33{50 (2006) 21. Rossi, F., Venable, K.B., Walsh, T.: A short introduction to preferences: between arti cial intelligence and social choice. Synthesis Lectures on Arti cial Intelligence and Machine Learning 5(4), 1{102 (2011) 22. Sanders, R.: Radioactive potassium may be major heat source in earth's core. UC
Berkeley News. Retrieved pp. 02{28 (2007) 23. Schlumberger: High temperature drilling operations. Tech. rep., Schlumberger: Oileld Services (2015) 24. Shoham, Y., Leyton-Brown, K.: Multiagent systems: Algorithmic, game-theoretic, and logical foundations. Cambridge University Press (2008) 25. The UK Oil & Gas industry association: Oil & Gas UK Economic Report 2017,
ISBN: 1903004942. Tech. rep., OIL&GASUK (2017) 26. Von Neumann, J., Morgenstern, O.: Theory of games and economic behavior.
Princeton university press (2007) 27. Xiushan, L., Shushan, H., Zhengchao, Z.: Hydrodynamic equations model mudpulse telemetry transmissions. Oil & gas journal 101(3), 47O49 (2003) 28. Yokoo, M.: Distributed constraint satisfaction: foundations of cooperation in multiagent systems. Springer Science & Business Media (2012)