<?xml version="1.0" encoding="UTF-8"?>
<TEI xml:space="preserve" xmlns="http://www.tei-c.org/ns/1.0" 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
xsi:schemaLocation="http://www.tei-c.org/ns/1.0 https://raw.githubusercontent.com/kermitt2/grobid/master/grobid-home/schemas/xsd/Grobid.xsd"
 xmlns:xlink="http://www.w3.org/1999/xlink">
	<teiHeader xml:lang="en">
		<fileDesc>
			<titleStmt>
				<title level="a" type="main">Software for indefinite integration Corrundum.red -Integration for all</title>
			</titleStmt>
			<publicationStmt>
				<publisher/>
				<availability status="unknown"><licence/></availability>
			</publicationStmt>
			<sourceDesc>
				<biblStruct>
					<analytic>
						<author>
							<persName><forename type="first">A</forename><forename type="middle">C</forename><surname>Norman</surname></persName>
							<affiliation key="aff0">
								<orgName type="institution">Trinity College</orgName>
								<address>
									<settlement>Cambridge</settlement>
									<country key="GB">U.K</country>
								</address>
							</affiliation>
						</author>
						<author role="corresp">
							<persName><forename type="first">D</forename><forename type="middle">J</forename><surname>Jeffrey</surname></persName>
							<email>djeffrey@uwo.ca</email>
							<affiliation key="aff1">
								<orgName type="department">Dept. Mathematics</orgName>
								<orgName type="institution">The University of Western Ontario</orgName>
								<address>
									<settlement>London</settlement>
									<region>Ontario</region>
									<country key="CA">Canada</country>
								</address>
							</affiliation>
						</author>
						<title level="a" type="main">Software for indefinite integration Corrundum.red -Integration for all</title>
					</analytic>
					<monogr>
						<idno type="ISSN">1613-0073</idno>
					</monogr>
					<idno type="MD5">8EB2D0A3C298091779DB4D1028F15169</idno>
				</biblStruct>
			</sourceDesc>
		</fileDesc>
		<encodingDesc>
			<appInfo>
				<application version="0.7.2" ident="GROBID" when="2025-04-23T18:34+0000">
					<desc>GROBID - A machine learning software for extracting information from scholarly documents</desc>
					<ref target="https://github.com/kermitt2/grobid"/>
				</application>
			</appInfo>
		</encodingDesc>
		<profileDesc>
			<textClass>
				<keywords>
					<term>Indefinite integration</term>
					<term>Anti-derivatives</term>
					<term>Rule-based Integration</term>
					<term>RUBI database</term>
					<term>Reduce</term>
					<term>Mathematica Mathematica</term>
					<term>Maple</term>
					<term>Mupad</term>
					<term>Maxima</term>
					<term>FriCas</term>
					<term>Giac/Xcas</term>
					<term>SymPy</term>
					<term>Reduce</term>
					<term>Rubi</term>
				</keywords>
			</textClass>
			<abstract>
<div xmlns="http://www.tei-c.org/ns/1.0"><p>The Rubi system for the evaluation of indefinite integrals was developed by Albert Rich over a period of more than 15 years. His unexpected death has led to a number of discussions on what can be done to develop the system further. One possible direction addresses the fact that Albert Rich's development was built entirely on the Mathematica system. A number of people have considered the porting of Rubi to a variety of alternative computer algebra systems. The requirements for porting are discussed.</p></div>
			</abstract>
		</profileDesc>
	</teiHeader>
	<text xml:lang="en">
		<body>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="1.">Introduction</head><p>For over 80 years, a key resource regarding indefinite integration has been Gradshteyn and Ryzhik <ref type="bibr" target="#b0">[1]</ref>. It has also been maintained and updated over the years, the 8th edition <ref type="bibr" target="#b1">[2]</ref> being released in 2015. The modern alternative is one of the computer algebra systems. These have made major progress since the 1960s, when Slagle's SAINT <ref type="bibr" target="#b2">[3]</ref> and Moses's SIN <ref type="bibr" target="#b3">[4]</ref> were released. The web site 12000.org <ref type="bibr" target="#b4">[5]</ref> reports testing 9 currently available systems 1 (commercial and public domain) against a test suite of 106812 integrals. As with any test suite, it can be criticized for showing biases, but it covers all the basic integrals and is of a scale that makes it hard to ignore. To a good approximation it is a concatenation of all the other significant sets of test cases that its authors could find.</p><p>The commercial systems have the highest success rates, with the open source systems significantly lower, save that the permissively licensed Rubi shows outstanding capability. We have therefore been studying Rubi to investigate the feasibility of its use with systems other than Mathematica, which is currently the foundation on which it is built.</p><p>We view this as being potentially valuable for four main reasons:</p><p>1. Many systems are still unable to find integrals for a significant fraction of the examples in the above torture test. This is likely to apply to almost every case that requires advanced special functions to express the result. Perhaps for many users this will not be a serious limitation, because their problem will not involve say the Fresnel 𝑆 function. But measured against Gradshtein and Ryzhik it is a limitation. 2. Where results are generated they may often be bulkier than would be ideal. This can go as far as returning a result expressed using elliptic integrals or incorporating imaginary numbers when something more elementary would serve better. 3. Something that may count as a special case of the above is the treatment of multi-valued functions and delivering an integral in a form that handles them and their principal values consistently. This can be of extreme important in integration when users substitute in endpoints to find a definite integral.</p><p>4. With reasonable generality, the existing schemes return integrals with no commentary available as to how they were derived, and the techniques used are embedded in large bodies of code which will be unavailable when one of the commercial systems is used, but opaque even if an open source system is selected. With Rubi a commentary on how results were obtained is almost automatically available.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="2.">What is Rubi?</head><p>The RUle-Based-Integration system Rubi is a public domain project addressing indefinite integration <ref type="bibr" target="#b5">[6]</ref>. It consists of two parts. The first part is a database consisting of transformation rules which convert an integral expression into a simpler form, allowing an iterative application of the rules to evaluate the initially given integral. The second part is a collection of utility functions, at present coded in the Mathematica language, which massages mathematical expressions into forms suitable for the application of one of the transformation rules contained in the database.</p><p>Readers not familiar with Rubi may appreciate a little background concerning the project. Rubi was started by Albert <ref type="bibr">Rich (1949</ref><ref type="bibr">Rich ( -2023) )</ref> circa 2007, after the development of the Derive computer algebra system was discontinued <ref type="bibr" target="#b6">[7]</ref>. It is based on a term-rewriting rule-based paradigm, which was expected to bring the following benefits.</p><p>• Speed and compactness.</p><p>• An ability to display the steps taken during an evaluation.</p><p>• Results which are correct in the complex plane.</p><p>• Results which are in the most compact and aesthetically pleasing form possible.</p><p>• A system which is readily modified and extended.</p><p>Although Albert Rich corresponded with many others, and received their suggestions for integration rules, he largely worked alone, and as a consequence, only lightly documented his program code.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="2.1.">The current state of Rubi</head><p>There are presently two versions of Rubi available. The first is 4.16.1; this is the last version whose release was overseen by Albert Rich <ref type="bibr" target="#b5">[6]</ref>. The other is 4.17.3, which has been retrieved from material left by Albert Rich and implemented by supporters of Rubi <ref type="bibr" target="#b7">[8]</ref>. The versions are very similar and can be summarized as follows.</p><p>• There is a database containing over 7000 rules, recorded in Mathematica syntax. It is in the public domain. They rules can be read in mathematical notation in PDFs from the Rubi website <ref type="bibr" target="#b5">[6]</ref>. • Rubi includes several test suites of integration problems. These form part of the test suite used by 12000.org. • The order in which the rules are placed can have an effect on the performance.</p><p>• The system consists of more than the database of rules. There are large support files, also written in Mathematica syntax, defining many auxiliary functions. These functions serve a variety of roles.</p><p>-Testing the properties of a parameter. For most integration problems, the parameters in the rules are given numerical values. Thus, it is assumed that a parameter can be tested to see whether it is positive, or an integer, and so on. -Applying an algebraic transformation, such as extracting the content of a polynomial, making a partial fraction expansion, etc.</p><p>Unfortunately, there are very few comment lines in the files containing the auxiliary functions. This makes working with them difficult.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="2.2.">The way forward</head><p>It is unlikely that a person or persons will come forward and take over the development of Rubi. The attitude of the de facto custodians of Albert Rich's legacy is that Rubi is frozen in its current state. A person wishing to continue Rubi is allowed to use the database as they see fit, but any development will be a new project. The analogy being made is that anyone can read a scientific paper and then write their own paper with whatever new thoughts the author has. The author references and acknowledges the earlier work, but is au fond working on a separate project. There are several ways in which Rubi can be developed further.</p><p>• More integration rules can be developed.</p><p>• There are a number of integrals in the test files which cause Rubi to disappear into a loop. A modification of the data base could reduce the occurrence of loops. • Rubi could be transported to other languages, such as Maple, muPad, etc.</p><p>For those of us who do not have Mathematica installed on our computer (perhaps because of cost), or who really value being able to inspect the workings of software whose results we rely on, or who are developers of other algebra systems (present and future), Rubi looks a very attractive starting point. As already mentioned, at present Rubi 4.x.x exists in the form of a large set of Mathematica rewrite rules and a significant but not-too-large collection of utility code to back them up. Albert Rich, however, had been planning a transition to Rubi 5 that would transform the rule-set into a decision tree that would not rely to the same extent on pattern matching in the system it was installed on. That was expected to make it much easier to migrate Rubi for use with other systems. Unfortunately this part of the project remains incomplete.</p><p>The work explained here is called "Corrundum.red" and is a step towards making full Rubi available outside Mathematica. While it is being developed using the Reduce system <ref type="bibr" target="#b8">[9]</ref>[10], the intent is to keep reliance on the algebraic facilities in Reduce fairly low and reasonable well explained, so that adaptation elsewhere might be more readily possible. In particular the rule-rewrite engine does not rely on any such engine already built into Reduce; thus exactly what it does can be made explicitly visible.</p><p>The name "corrundum.red" is of course something of a pun, in that corrundum is crystalline aluminium oxide with hardness 9 on the Mohs scale, and when tainted with chromium so that it shows up as a vivid red colour you obtain the gemstone ruby. Of course ".red" is the file suffix used by the Reduce algebra system. So this name references Rubi (which is a sometimes-used alternate spelling for ruby); it links to Reduce and is intended to suggest both toughness and high value.</p><p>There have been multiple exercises to leverage the Rubi rule set in the past, so we are not the first to look into this. In fact, a previously abandoned investigation, using Reduce, was conducted by one of us (ACN).</p><p>We of course note the existence of "symja" or 𝑆Ψ𝑀 𝐽Λ which aims mainly at support for Android and is all coded in Java. That has at its core an implementation of a language rather closely modelled on the one used by Mathematica, and as a result they have been able to import Rubi wholesale leading to very impressive capabilities. For our purposes this provides excellent confirmation that emulating enough of Mathematica to make Rubi work is feasible, and it provides a non closed-source version. But the Mathematica-style support there pervades so much of Symja that it does not help to guide those who want only enough of it for use in some alternative system that does not start off in Java and does not start off with Mathematica-like syntax and semantics baked in.</p><p>Especially with Albert Rich's work migrating Rubi from a rule-based system to one making transitions based on a decision tree, significant parts of Rubi were made to work on Maple, however again that project seems never to have reached a stage of full support for all the rules. https://de.maplesoft.com/company/casestudies/stories/maple-for-accuracy-and-support.aspx?L=G Similarly Julia has the adoption of Rubi on its 2021 roadmap: https://juliasymbolics.org/roadmap/ but the github repository that we found https://github.com/ufechner7/Rubi.jl has been inactive for some years.</p><p>With all this background our work has many precedents. We hope that this paper will explain, in more detail than we were able to find elsewhere, some of the particular challenges in this activity, while we attempt to extricate Rubi from Mathematica syntax, semantics and support, and thus be of interest to others who want to revive previous attempts or start new ones.</p><p>Corrundum.red has a number of components and these will be explained in the following sections.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="3.">Capturing the Rubi rules</head><p>A parser for the subset of Mathematica syntax used in the transformation rules was coded for Reduce.</p><p>Happily a rather naive recursive descent parser proved sufficient. Anyone at all familiar with this style of parser would observe that it is basically merely one procedure to correspond to each entry in the BNF rendition of the syntax involved. This naive parser was first created a decade ago and used to start an investigation of a version of Rubi from back then: that project stalled and the revived parser now had to have some extensions to cope with the Mathematica syntax that has meanwhile crept into Rubi and its utilities. Over that decade Reduce has been provided with a parser generator using the standard LALR scheme which could have made this scheme for reading Mathematica style text easier and much more compact, but re-using the existing almost-working if cruder scheme seemed the simpler path. And of course since this is just parsing expressions and not a full language the naive approach is not seriously offensive. We believe it would also have been possible to make Mathematica export files in a very similar format. The result of a parse is the full set of rules expressed as Lisp-style data structures with a collection of operators that denote the various things that Mathematica provides. So for instance the very first Rubi rewrite starts as </p><formula xml:id="formula_0">(expt (_ x) (_. n)))) (_. p))) (_ x Symbol)) (Int (times u (expt (times b (expt x n)) p)) x)) (and (FreeQ (bracelist a b n p) x) (EqQ a 0)))</formula><p>Actually in the file as generated all punctuation characters in the parenthesised form are preceded by escape characters so that for instance "/;" and "_. " are seen as merely simple symbol names.</p><p>This adjusted format explicitly represents a parse tree and it is likely that internally Mathematica turns the raw input into something equivalent -but what we have here is totally unambiguous and entirely suitable for processing with any scheme that is happy to work with tree-like data structures. At the data structure level Reduce works with Lisp-style data and so it is especially comfortable with this, but it would be close to trivial to write code to read this tree notation in almost any language. Given that a simple tree-walk could re-display the material in the syntax most suitable for some other system, be it re-creating Mathematica syntax or converting for Maple, Maxima, Axiom or whatever.</p><p>It can be seen that a rule generally has three components. A template or pattern which here is basically ∫︀ (𝑢 * (𝑎 + 𝑏𝑥 𝑛 ) 𝑝 , 𝑥) and where the various annotations are to mark various things as parameters or wildcards. Then there is the transformed version, which in this case purely serves to remove the 𝑎 that is originally present. Finally there is a set of conditions. These indicate that the various parameters must all be independent of 𝑥 and that this transformation that gets rid of 𝑎 is only applicable when 𝑎 = 0, in this case reminding us that the actual expression may contain a sub-expression equivalent to 0 but starting off looking more complicated. Of course we all know that identifying when a general expression is zero is undecidable! There are almost 7500 of these rules! Using them involves facing up to (at least) two significant challenges. One is that of gaining a sufficiently full understanding of Mathematica pattern matching so that its behaviour against this rule-set can be re-created. The other is that throughout the Rubi rules there are function calls in both conditions and replacements that invoke a range of Mathematica primitives. Some (such as FreeQ here) have behavior that is simple to understand, but the precise expectations and limitations of even something as obvious looking as EqQ[a,0] have to cause some pause for thought.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="4.">Pattern matching and the Utilities</head><p>When one first admires Rubi, what most catches the eye is the set of simple-looking tree rewrite rules. These basically look at the structural form of integrands and on that basis perform transformations. Furthermore it is on record that Albert Rich's plans (for a Rubi 5) involved transforming the identification of which rules to apply into a decision tree -in effect a giant nest of "if" statements.</p><p>Thus when we started work, in a spirit of optimism we expected that, with the rule-set expressed as data structured such that Reduce could manipulate it easily, our main task would be to get simple tree-rewrites involving 7000+ rules to run reasonably efficiently.</p><p>After some while it became clear that things were not quite that straightforward.</p><p>1. When one writes Int[expression, x] in Mathematica, the expression delivered by Mathematica to pattern matching definitions of Int is neither the unaltered input nor a fully simplified version of it. It is somewhere in between. If Rubi is ported to any other world, this pre-processing is not likely to be replicated precisely, and so behaviour will differ. As a tiny example consider the Mathematica definition f[a_*b_] := {"product", a, b} which decomposes a product. The input f <ref type="bibr">[x+x]</ref> reveals that what Mathematica sees for matching is the product of 2 and x, while f[x*x] is not seen as a product at all but as a power. f[(1+x)*(x+1)] is again seen as a power, while the more elaborate f[x*((x+1)^2-1-x^2)] is not seen as having an argument that is equivalent to 𝑥 2 and so is reported as being product. All of this is perfectly proper to the extent that it is Mathematica doing what it chooses to do, but it does make it harder to think about testing Rubi, since the exact form of expressions it sees have been subject to this not very clearly documented adjustment. 2. Mathematica pattern matching is powerful and it is aware that addition and multiplication are both commutative. Thus a pattern of the form Int[a*x^2 + b*x + c, x] (here the underscores are omitted to make the presentation neater) will match a quadratic regardless of the order in which the terms are presented. So in some sense a rule with that pattern as its left-hand side represents 6 cases corresponding to different term orderings. At one stage we considered preconditioning the rule set so that all of those cases were explicitly shown and hence the actual matching would not need to worry about this issue and hence might be simpler and faster: it became apparent that doing so caused the size of the rule set to explode beyond reason. 3. A further cleverness of the Mathematica matcher is that in certain cases some parameters can have default values. This is indicated by using "_." where they are mentioned in a pattern. With this scheme the pattern a_. + b_.*x_^n_. will match not just p+q*z^k but also variants on that with p=0 and/or q and k=1, all the way down to just z. If one combines the consequence of this with commutativity this one pattern can match a dozen different input forms including say z*q+p. One rewrite rule can contain multiple instances of these two shorthands and we found that if everything was expanded out to show the full range of cases to be accepted that we could end up with well over a thousand cases following from what was presented as a single rule. This astonished us! 4. The matching that is made has to be subject to some conditions -which are given at the end of the rule following the symbol /;. A first thought is to perform structural matching first and after that check if the constraints apply. However consider a case where the pattern is (without the underscores here) a + b*x^n + c*x^m and the constraints include m=2*n. Now commutativity shows that the two sub-patterns b*x^n and c*x^m could match input in either order, but only one is will suit the constraint. The most naive approach would be to insist that the syntactic matcher returned not just a single match but all possible ones that arise based on commutativity and default values and that the constraints are then used to determine which (if any) are viable. In some cases this would return very many matches. It feels more reasonable to make the structural matcher able to deliver its first match and then backtrack to look for another if that becomes necessary. Of course in the worst case just as many variants could have to be tried.</p><p>There are ways to address all of these issues. The pre-conditioning'of input for instance to sort constant terms in sums and products either to the start or the end helps substantially with commutativity. Where we have arrived at is that we index constraints by the parameters they depend on. Then when our pattern matcher is about to ascribe a value to a parameter it checks if that completes the information about everything that the relevant constraints will use. It can thus test the conditions during matching and as soon as possible, and this of course saves us from investigating parts of an expression once we have discovered that that will be futile. With this we then accept the first viable ordering for any commuting argument list.</p><p>We are aware that our early-check scheme could be unsatisfactory for a fully general set of rules. Consider a complete pattern that contains both (x^n+x^m) and (x^p+x^q) and then pathologically a constraint on n, m, p and q that is not symmetric in them. The orderings for matching the two commuting sums would need to coordinate. We believe there are no such situations in the Rubi rule-set, so our matcher is not fully general but should cope with requirements here. This illustrates that an implementation of Mathematica-like matching that just has to cope with a fixed set of rules such as those in Rubi may be able to be simpler and hence potentially faster than one that needs to come with the most general case.</p><p>The matcher we have coded is essentially a modest size body of Lisp code and of itself it is independent of Reduce. It is intended to be reasonably clear and as concise as we can manage, so for instance to cope with a commutative operator it forms a list of all permutations and then scans the list reporting the first occasion it finds a match. It would clearly be possible to fold the enumeration of permutations with checking them but for a first version and for "work in progress" we preferred simplicity over potential performance gains. In a similar way and although we have thought of many ways that will allow us to do better, we have started with checking input expressions against each of the 7000+ patterns in turn and we have avoided exploring the interesting rabbit-hole of indexing, expression signatures and hashing, "trie" structures and expansion of the matching process into executable code. Those are all for further investigation at a later stage.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="5.">Predicate and Replacement material</head><p>On starting this project we expected that pattern matching would be the core of the work. Well, the conditions applied to rules contain many predicates that are obvious and easy to support. For instance for a very large proportion of parameters there is a check that the parameter does not depend on the independent variable. For exponents there can be checks such as that 𝑛 is not −1. The test 𝑚 = 2 * 𝑛 is slightly less obvious but once encountered is easy to handle. So at first we felt we were making good progress as we provided more and more of those. Within the formulae that appear as results there are cases where 𝑛 is numeric and 𝑛 + 1 is used, and it makes obvious sense to perform the arithmetic. Again that was easy. Perhaps especially so as it plays to Lisp's strengths and is all code that could reasonably have appeared in 1960s attempts at integration! Following this path and by providing a tiny set of our own integration rules we were able to perform some first integrations really rather early in our work. At that time we felt greatly encouraged.</p><p>Beyond the trivial there are then a collection of Mathematica operations that do "genuine" algebra, but where the intent is clear and where any other host algebra system will have something equivalent. An example arises when a result will be of the form Log[expression] because if the expression has a constant factor (i.e. a non-trivial content with respect to the variable of integration) then that can be taken out and its only contribution to the result would be to merge it in with the constant of integration. So when Rubi uses a function RemoveContent[] that is liable to involve chaining to the underpinning algebra system and the exact details of how that is done can not be portable but the expected returned value is unambiguous. Some Mathematica primitives as called here can of course represent special portability challenges and in the end the balance as to how much of Rubi's success flows from its own rule-set and how much from the power of (say) the Mathematica Simplify[] function (which will apply multiple transformations on its input and return the result that Mathematica things is nicest) is something that can not be answered until our work is complete.</p><p>However we then found that in both predicates and especially in replacements there were instances of function calls with significant depth. These are defined in a file IntegrationUtilityFunctions.m. The around 8000 lines of Mathematica code there include a great many small wrappers for simple operations that are not problematic at all, however there are other sections that gave us pause. An example is the function Subst which is fairly cryptically explained there as "Subst[u,x,v] returns u with all nondummy occurrences of x replaced by v and resulting constant terms replaced by 0." but where inspection reveals that it together with its immediate sub-functions represents around 800 lines of code. For a while that felt like a really severe road-block.</p><p>Our eventual response has been to recognize that it is necessary to view the Utilities file as a further set of rewrites to be handled alongside and using basically the same mechanisms as the main set of integration rules. This is possible because much of Mathematica's notation is close to being functionalall the "procedure definitions" in the Utilities file can, from only a slightly different perspective, be seen as "rewrites".</p><p>The Utilities use a distinctly broader range of Mathematica syntax and facilities than the main set of integration rules. This includes the various scoping constructs that interact with just how Mathematica interprets the meaning of symbols, catch and throw, mapping over lists with what are in effect lambda expressions and explicit in-line invocations of the pattern matcher. We have taken a pragmatic approach -when some utility function uses "fancy" Mathematica features but is in fact fairly small or simple we have just implemented a replacement in Lisp/Reduce so only large and messy functions need to be interpreted from their Mathematica form. We end up with what amounts to an interpreter for just enough of the Mathematica language to cope with Rubi, and a scheme where any particular function might end up either recursing in the interpreter to continue processing Mathematica forms or dropping out into our own independent code.</p><p>As we were working on that we had arranged that any attempt to use a function that had not yet been provided would lead to a clear message but that processing could continue (with Rubi typically just considering the rewrite concerned inapplicable and going on to try the next). This meant that by running the full almost 80K test cases through our code we could count which functions were blocking progress most often. At all stages this provided us with a form of priority list. For finer grain investigation we obviously also maintained scripts to test just a sample from the full test suite and also to try our own hand-picked cases so that they could be run with extra tracing.</p><p>For longer than we had imagined would be the case our corrundum.red could only deliver results for around 1% of the examples in the test suite and we perhaps started to feel a little despondent. On implementing some more Mathematica functions to get the Utilities going at one stage boosted that so we could solve almost 7% of the first 10000 test cases. The current state is that our code now does not report any dangling unimplemented functions but clearly the emulation of some of Mathematica is still buggy and for most examples the code yields a result that is a malformed formula. This is liable to be "just a bug" and so we hope it will soon be fixed! But this paper is explicitly about "work in progress" and the comments here show that that is exactly where we are.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="6.">Current Results</head><p>On testing cases from the Rubi test suite our performance so far is quite lamentable, with the very best try to date managing to get proper answers for just under 7% of the cases! But this mainly reflects some residual Mathematica primitives that have not yet been coded and debugged enough to let all examples run to completion. However our code can show the transformations that it makes and so one can observe the rewrite engine both when behaving well and when not.</p><p>Here is a case of mixed gladness, where there is some degree of mixture between Mathamatica botation (eg Log) and Reduce (eg log): "xtimes" is our rendering of the symbol ∖[Star] that Rubi introduces in some places where we suspect it is trying to prevent Mathematica from transforming products under its feet. You can see that at present we have not turned it back into ordinary multiplication on the way out. But you can also see that along the way Corrundum has used two of the Rubi rules (which we call numbers 2778 and 2779, those numbers being easily decoded in our source files) and that despite its ugliness our result here is not much bulkier than the "official" Rubi one and is basically correct. This and many other cases illustrates that the exact form of the official Rubi result depends on what amounts to post-Rubi term reordering and cleanup done by Mathematica, because in those cases our results can be equally correct but just (for instance) order terms differently.</p><p>Various of the more severe problem cases we face are going to be easy to track and trace from the sort of output we generate: first we show the input expression, then for each pattern that matches we document both the identity of the rule triggered and the resulting transformed expression. Given that this is all basically Lisp code it is then simple to set tracing on the functions that implement each step.</p><p>The above statement is correct in that debugging steps that are taken can be straightforwardhowever there may also be instances in which Corrundum fails to make a match that Rubi expects and perhaps relies on. This could either be because of limitations in our pattern matching code or because intermediate transforms have not left an expression in exactly the shape that Rubi expectedfor instance because reaching that shape was a consequence of fine details of Mathematica processing.</p><p>To help us track issues of that sort we are taking a second track. We have taken the full set of Rubi rules in Mathematica format and added annotations that cause Mathematica to report on each rewrite it makes, as in:  <ref type="figure">--------------+ --------------</ref>27 45</p><p>and by cross referencing this against output from our system we will be able to track rewrites that we do not make as well as debug ones that we do. Again the numbers attached to rules are ones that have to be interpreted by indexing into our versions of the source files -they are not absolutely fixed to the reference version of the Rubi sources. Also the rule-tracing is not the one that Rubi has as one of its nicer features -it is one that provides yet finer grain information about the rewrites that are applied.</p><p>This sort of trace also makes it possible to identify the modest number of cases where the current Rubi rule-set can lead to unending cycles of transformation, so that attention can be given to tidying up there. It also has the potential to let us identify both which Rubi rules are used most (so we can improve performance by checking for those ones first) and perhaps spotting ones that are never used at all.</p><p>As has been notes several times already this is a project still in full swing, but we believe it has already hit a number of valuable goals:</p><p>1. In understanding the current state of Rubi; 2. Understanding much more about the way in which it intertwines with Mathematica than any experiment that remained within the Mathematica world could; 3. Provide a somewhat freestanding rewrite engine to use the Rubi rules. The dependence we have on Reduce for algebra support is fairly modest (at least in our opinion) and (maybe when we get round to it) easy to document. The main engine we have is expressed as Lisp code (albeit presented in Reduce syntax that sugars it and may make it easier for outsiders to come to grips with it; 4. Get a system that can perform some integration using the Rubi rule-set, with a real prospect that getting a lot further is now "just debugging". We have a "proof of concept". 5. As we have developed this we have stashed away multiple ideas for performance engineering to make our system rather fast.</p><p>So this has been and remains an entertaining project to have been working on, and we have now reached a state where we would be willing to get others with suitable coding competence to join in to push the work further, either until we have something fit to merge into Reduce or to build on what we have done to consider support for say Maxima, Maple or some other system.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="7.">Some lessons mainly about Mathematica that we have learned</head><p>The characterisation here comes from the perspective of one who is unfamiliar with Mathematica and only confronts it through the prism of Rubi. This will of course provide a tinted and distorted impression, but it is nevertheless relevant while trying to reproduce Rubi's behaviour. So here we collect points, some of which have been alluded to earlier but are still things we know now that we did not at the start of this work.</p><p>To a first approximation Mathematica has two schemes for processing algebraic expressions. There seems to be a range of transformations that are automatically and universally made. These include cases like 1+1 → 2 but also 𝑎*𝑧 *𝑎 → 𝑎 2 *𝑧 where terms in products are sorted and consolidated into powers when they are sufficiently identical, but not if they are equivalent but sufficiently textually different. There are then a wide range of functions that can explicitly perform operations such as expanding products, arranging a common denominator and so on. Finally there are generic simplification functions that try out a larger or smaller number of the explicit transformations and return what they judge to be the nicest variant on their input that they come across.</p><p>We explain this here because a study of the Rubi sources shows clearly that Albert Rich found the need to work around some of the details of all this. Two cases in particular illustrate this by showing transformations only reasonably comprehensible as work-arounds to detailed Mathematica behaviour, In Mathematica the standard trigonometric functions are named Sin, Cos, Tan and the like -with upper case initial letters. The Rubi rules and utilities in places introduce a parallel set called sin, cos and tan in lower case that are referred to as "inert". Rules first convert from the standard Mathematica versions to these ones and apply their own set of simplifications, but also at least potentially pass the inert expressions through more general Mathematica transformations where presumably otherwise the system would have "simplified" things in unwanted ways.</p><p>Somewhat similarly ∖[Star] is used for a variant on multiplication which is processed a little differently from the regular Mathematica one.</p><p>Finally the utility functions file contains a definition where SetAttributes seems to be being used to make fairly broad adjustments to Mathematica processing in a way that anybody not familiar with both all the Rubi rules and their intended interactions and just what Mathematica will do with or without that directive may find challenging to decode. A feeling that arises from all of this is that to support Rubi fully outside Mathematica it is perhaps necessary to emulate some of the corners of Mathematica behaviour where Rubi is then carefully taking steps to allow for the fact that it did not want them to apply! None of this is to be viewed as a criticism of either Mathematica or of that state of the final Rubi snapshot, but it may suggest that the entanglement between them is non-trivial. A different view is that perhaps in reality the worries expressed here will end up impacting only a small fraction of the examples in the test suite, and in some future Rubi some adaptation or adjustment of rules may sort that out easily. Until our work is complete we can not tell!</p></div><figure xmlns="http://www.tei-c.org/ns/1.0" xml:id="fig_0"><head>(</head><label></label><figDesc>* ::Code:: *) Int[u_.*(a_+b_.*x_^n_.)^p_.,x_Symbol] := Int[u*(b*x^n)^p,x] /; FreeQ[{a,b,n,p},x] &amp;&amp; EqQ[a,0] plus (_ a) (times (_. b)</figDesc></figure>
<figure xmlns="http://www.tei-c.org/ns/1.0" xml:id="fig_1"><head></head><label></label><figDesc>Test case 57 Use rule 2778 on Int(Log(x),x) transforms to x*Log(x) + (-1)*x Use rule 2779 on Int(Log(x)^2,x) transforms to x*Log(x)^2 + (-1)*xtimes(2,x*Log(x) + (-1)*x) (~~~test case 57 My leafcount 19 Theirs 15) Input: int(log(x)^2,x) My result: x*log(x)^2 + (-1)*xtimes(2,x*log(x) + (-1)*x) Reference result from Rubi test file: 2*x + (-2)*x*log(x) + x*log(x)^2</figDesc></figure>
<figure xmlns="http://www.tei-c.org/ns/1.0" xml:id="fig_2"><head>Test case 2 :</head><label>2</label><figDesc>Int[x Sqrt[1 + 3 x], x]</figDesc></figure>
<figure xmlns="http://www.tei-c.org/ns/1.0" xml:id="fig_3"><head>FixIntRules</head><label></label><figDesc>[rulelist_] := Block[{Int, Subst, Simp, Star}, SetAttributes[{Int, Subst, Simp, Star},HoldAll]; Map[Function[FixIntRule[#,#[[1,1,2,1]]]], rulelist]]</figDesc></figure>
		</body>
		<back>
			<div type="references">

				<listBibl>

<biblStruct xml:id="b0">
	<monogr>
		<author>
			<persName><forename type="first">I</forename><forename type="middle">S</forename><surname>Gradshteyn</surname></persName>
		</author>
		<author>
			<persName><forename type="first">I</forename><forename type="middle">M</forename><surname>Ryzhik</surname></persName>
		</author>
		<title level="m">Table of Integrals, Series, and Products, Gosudarstvennoe Izdatel&apos;stvo Tehniko-Teoretieskoj Literatury</title>
				<imprint>
			<date type="published" when="1943">1943</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b1">
	<monogr>
		<author>
			<persName><forename type="first">I</forename><forename type="middle">S</forename><surname>Gradshteyn</surname></persName>
		</author>
		<author>
			<persName><forename type="first">I</forename><forename type="middle">M</forename><surname>Ryzhik</surname></persName>
		</author>
		<author>
			<persName><forename type="first">D</forename><surname>Zwillinger</surname></persName>
		</author>
		<author>
			<persName><forename type="first">V</forename><surname>Moll</surname></persName>
		</author>
		<ptr target="https://cds.cern.ch/record/1702455.doi:0123849330" />
		<title level="m">Table of integrals, series, and products</title>
				<meeting><address><addrLine>Amsterdam</addrLine></address></meeting>
		<imprint>
			<publisher>Academic Press</publisher>
			<date type="published" when="2015">2015</date>
		</imprint>
	</monogr>
	<note>8th ed</note>
</biblStruct>

<biblStruct xml:id="b2">
	<monogr>
		<author>
			<persName><forename type="first">J</forename><surname>Slagle</surname></persName>
		</author>
		<title level="m">A heuristic program that solves symbolic integration problems in freshman calculus : symbolic automatic integrator</title>
				<imprint>
			<date type="published" when="1961">1961</date>
		</imprint>
		<respStmt>
			<orgName>MIT</orgName>
		</respStmt>
	</monogr>
	<note type="report_type">Ph.D. thesis</note>
	<note>SAINT)</note>
</biblStruct>

<biblStruct xml:id="b3">
	<monogr>
		<author>
			<persName><forename type="first">J</forename><surname>Moses</surname></persName>
		</author>
		<idno>AC-TR-47</idno>
		<title level="m">Symbolic Integration</title>
				<imprint>
			<date type="published" when="1967">1967</date>
		</imprint>
		<respStmt>
			<orgName>MIT</orgName>
		</respStmt>
	</monogr>
	<note type="report_type">Technical Report</note>
</biblStruct>

<biblStruct xml:id="b4">
	<monogr>
		<title level="m" type="main">Cas integration tests</title>
		<author>
			<persName><forename type="first">N</forename><forename type="middle">M</forename><surname>Abbas</surname></persName>
		</author>
		<ptr target="https://www.12000.org/my_notes/CAS_integration_tests/index.htm" />
		<imprint>
			<date type="published" when="2024">2024</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b5">
	<monogr>
		<author>
			<persName><forename type="first">A</forename><forename type="middle">D</forename><surname>Rich</surname></persName>
		</author>
		<author>
			<persName><forename type="first">P</forename><surname>Scheibe</surname></persName>
		</author>
		<ptr target="https://rulebasedintegration.org/" />
		<title level="m">Rule-based Integrator</title>
				<imprint>
			<date type="published" when="2024">2024</date>
		</imprint>
	</monogr>
	<note>Rubi</note>
</biblStruct>

<biblStruct xml:id="b6">
	<analytic>
		<title level="a" type="main">A knowledge repository for indefinite integration based on transformation rules</title>
		<author>
			<persName><forename type="first">A</forename><forename type="middle">D</forename><surname>Rich</surname></persName>
		</author>
		<author>
			<persName><forename type="first">D</forename><forename type="middle">J</forename><surname>Jeffrey</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="m">Intelligent Computer Mathematics -LNCS 5625</title>
				<imprint>
			<publisher>Springer</publisher>
			<date type="published" when="2009">2009</date>
			<biblScope unit="page" from="480" to="485" />
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b7">
	<monogr>
		<author>
			<persName><forename type="first">A</forename><surname>Rich</surname></persName>
		</author>
		<author>
			<persName><forename type="first">P</forename><surname>Scheibe</surname></persName>
		</author>
		<ptr target="https://github.com/RuleBasedIntegration/Rubi/releases/tag/4.17.3.0" />
		<title level="m">Rubi -rule-based integration</title>
				<imprint>
			<date type="published" when="2024">2024</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b8">
	<analytic>
		<title level="a" type="main">REDUCE: A user-oriented interactive system for algebraic simplification</title>
		<author>
			<persName><forename type="first">A</forename><forename type="middle">C</forename><surname>Hearn</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="m">Interactive Systems for Experimental Applied Mathematics</title>
				<editor>
			<persName><forename type="first">M</forename><surname>Klerer</surname></persName>
		</editor>
		<editor>
			<persName><forename type="first">J</forename><surname>Reinfelds</surname></persName>
		</editor>
		<meeting><address><addrLine>New York</addrLine></address></meeting>
		<imprint>
			<publisher>Academic Press</publisher>
			<date type="published" when="1968">1968</date>
			<biblScope unit="page" from="79" to="90" />
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b9">
	<monogr>
		<author>
			<persName><forename type="first">A</forename><forename type="middle">C</forename><surname>Hearn</surname></persName>
		</author>
		<ptr target="https://sourceforge.net/projects/reduce-algebra" />
		<title level="m">many contributors, Reduce -a portable general-purpose computer algebra system</title>
				<imprint>
			<date type="published" when="2024">2024</date>
		</imprint>
	</monogr>
</biblStruct>

				</listBibl>
			</div>
		</back>
	</text>
</TEI>
