Intelligible Intelligence: Deep XAI still more R&D than toolbox

Most architectural tradeoffs are hard. So is the one between the accuracy of Deep Machine Learning (ML) and explainability/transparency of explainable AI (XAI). Therefore, DARPA’s initial XAI program is expected to run through 2021.

Few explainability mechanisms have been extensively tested on humans, but current R&D indicates at least some (hybrid) tech to come in a couple of years.

Several well-tried ML technologies work their way through (in steps) from a first random solution to satisfactory ones, and where possible, to an optimal one:

·         Genetic Algorithms, by creating additional generations

·         Forests, by creating additional trees from the same data

·         Deep Neural Networks, by a (cost) function applied (in training) to outputs, based on how they differ from labeled data, and propagated back across all neurons/synapses, to adjust weights

·         Hybrids, by combination (the “new kid on the block”).

NASA Space Technology 5 antenna created by GA (source: NASA.gov)

GA and Forests are more than semi-intelligible, by their nature. However, now that deep learning feeds big data through NN that consist of multiple hidden layers of neurons (DNN), it arrives at very accurate solutions to complex multidimensional problems, but at the same time at an inherent black box.

A part of my post from August is about random forests, which offer both more transparency than DNN do plus quite a degree of accuracy. Moreover, trees and NN are even cross-fertilized, to offer explainability without impeding accuracy, and there are already several flavors of this; to pick a handful:

·         adding extraction of simplified explainable models (e.g. trees) onto black-box DNN

·         local (instance-based) explanation of one use case at a time, with its input values, for example animating & explaining its path layer-by layer (like most test tools do).

·         soft trees, with NN-based leaf nodes, that perform better than trees induced directly from the same data

·         adaptive neural trees and deep neural decision forests that create trees (edges, splits, and leafs) , to outperform “standalone” NN as well as trees/forests that skip the combination.

 

Explainability, transparency, and V&V are absolutely essential to users’ reliance/confidence in mission-critical AI. Therefore, whichever path or paths take us to up-and-running products, XAI is welcome.

by Milan Kratochvil

Trainer at Informator, senior modeling and architecture consultant at Kiseldalen’s, main author: UML Extra Light (Cambridge University Press) and Growing Modular (Springer), Advanced UML2 Professional (OCUP cert level 3/3). Milan and Informator collaborate since 1996 on architecture, modelling, UML, requirements, rules/AI, and design. You can meet him at public courses (in English or Swedish) on AI, Architecture, and ML (T1913 , in December or February), Architecture (T1101, T1430) or Modeling (T2715, T2716).

Läs mer

Twin examples of multiple trees: 1. UML models, 2. Machine Learning

“Today, professionals get trained in using tools…  there’s a lack of education of fundamentals like modeling, architecture, methods, or concepts... Getting value out of data needs professionalization based on education and practical experience.”                                                                 

Andreas Buckenhofer, Daimler TSS, in an interview by OODBMS on Big Data, July 2019 

In my opinion, he’s spot on.

My post from March mentions why new AI languages aren’t exactly heavies of a CV in a mainstream business; in April, a figure (at the end of the post) also touched on Forest structures in ML and eXplainable AI.

After a free-wind sail that took us from detail to architecture, we now go into some structural “forestry”. It’s about tackling the same domain from multiple viewpoints, instead of clinging on to one.

1. Multiple trees in UML: Generalization sets

This post from 2015 (in Swedish) discusses the sets in more detail, so let’s just recap the diagrams, in English, and add «powertype» on the fourth one (a power set’s instances are subsets, so by the same token, a UML2 powertype’s instances are “subtypes” of a general construct).

2. Multiple trees in Machine Learning: random Decision Forests

Here, a designer is not the one who maps out subclasses. Rather, an ML algorithm generates in training time, from (labeled) data, an ability to perform classification (i.e accurate “mapping-out” of “classes”). The decision nodes of a (classification) tree gradually subdivide the data into more and more fine-grained classes.

Why bother about decision trees when Deep Neural Networks are booming? Because explainability opens the door to acceptance in mission-critical apps. ML-generated logic has to be auditable. User enterprises are pushing for graphicness, conceptualization, traceability, V&V. Those are the strengths of decision trees, and weaknesses of Deep NNs (we know those work, but hardly how); same thing with learning time required, size of training data sets, execution speed, or partitionability (a hint for IT architects: a tree works independently, whereas a neuron relies on many other ones). Atop of that, decision trees offer a structural backbone of hybrid AI systems (see also the last paragraphs in this post).

You might remember that trees in woods sometimes fuse their roots and exchange materials. Unsurprisingly, we find some synergy in virtual forests too. Firstly, our trees are “grown” on a random sample each (hence some “biodiversity” too), from one training-data set (hence fewer terabytes of training data). Secondly, on each sample, its tree’s decision nodes use a random subset of all available attributes. This gives architects and other roles some room to tune the mix of efficiency and explainability; in forests, it’s is near the level of genetic algorithms (GAs too have possible “mix-tuning points” in “biodiversity steps” Crossover and Mutation).

The more trees and “biodiversity” our ML algorithm grows, the more accurate and robust the generated logic becomes, because the final step is vote counting. A forest’s output (a classification like here, or a forecast) is an aggregated value of the outputs of all trees (a statistical mode in classification, or a mean in regression). It prevents the random decision forest from getting stuck in local optima, that is, we minimize error rates and overfitting to a given training-data set (which may be both incomplete and biased).

by Milan Kratochvil

Trainer at Informator, senior modeling and architecture consultant at Kiseldalens, main author : UML Extra Light (Cambridge University Press) and Growing Modular (Springer), Advanced UML2 Professional (OCUP cert level 3/3).

Milan and Informator collaborate since 1996 on architecture, modelling, UML, requirements, rules, and design. You can meet him in September at public courses (in English or Swedish) on AI, Architecture, and ML (T1913), Architecture (T1101, T1430) or Modeling (T2715, T2716).

Läs mer

Leveled up: Auditability of AI and Machine Learning

Architects and many others remember the tightrope walk between flexibility/performance and testability/predictability/V&V in systems with many run-time parameters, or parallelism, or late binding time ranging from polymorphism to SOA-UDDI and ad-hoc computing. Now, it’s leveled up by ML (machine learning). Essentially the same tradeoff, but growing broader and trickier. 

Black box 

ML stirs up the fire; despite its roots (rule induction and mining) in the successful decryption of an unbreakable cipher, its near future looks encoded in weight values somewhere in deep neural networks. Whereas black-box flight recorders clarified the chain of events & decisions in past emergencies, more and more IT is now landing in black boxes that hide opaque logic.

Predictability wasn’t a big deal in consumer IT and entertainment (when Youtube or Spotify wrongly offered you a title you were avoiding like the plague, you rarely asked why)… If you just say “skis this wide apart look amusing”, I guess you’re in consumer IT, but if you insist on a layer-by-layer explanation why most artificial vision systems have a hard time in strong sunshine on white slopes, I bet you’re in corporate (image: Ski Robot Challenge, Korea). 

Tackle it one-way or two-way

Business apps are very different from apps for billions of consumers (see slides 9 to 15 in this talk by Oracle’s VP at SICS). Your enterprise or team can tackle the leveled-up tradeoff both top-down and bottom-up:

• assuring a framework of corporate values and procedures (particularly transparency, governance & compliance, accountability, and a security & safety culture)

• applying appropriate technologies and practices in IT to build in mechanisms upfront  for auditability, comprehensibility, predictability, traceability, testability/V&V (as well as fraud-prevention, such as restricted access to learning-data sets).       

On the latter (bottom-up) part, there’s ongoing AI research to “unpack” the opaque logic buried within deep learning systems, and to give them an ability to explain themselves. DARPA’s Explainable AI Program, XAI , aims at ML techniques (new or improved) that produce more explainable models, while maintaining a high level of prediction accuracy. New machine-learning systems will have the ability to explain their rationale, strengths, weaknesses, etc.

Hybrid-AI tech vendors often address organizations with more constrained schedules, budgets and levels of AI expertise. Hybrid learning systems combine “subsymbolic” ML with transparent symbolic computation (typically, wellknown knowledge-processing techniques). The combination lowers the total cost of entry into AI and ML, because it evolves from logic that domain experts already know (rules, decision trees, etc.)

From there, hybrid systems employ ML iteratively to fine-tune this explicit logic: for example, to narrow the IF-part of a rule to factors that prove most significant. That is, results of ML from big data decide about variables to be included (or omitted), about intervalization of a continuum of values, or about relevant threshold values of a particular variable.

Notably, a rule is still expressed as a rule yet with an ever-smarter and more accurate IF-part. This is transparent to humans, and paves the way to embedding AI and ML into daily IT-dev practice: devs and architects will gradually find thousands of decision points, enterprise-wide, suited for small AI apps in daily business. Those will generate valuable skills, know-how, and “tip feeling” as to where ML can work (or can’t).

Models, animations, transparency

Once the opaque logic is unpacked, or expressed as rules or trees, it’s time to revive your team’s modeling skills. Long story short, a decision tree (or an invocation path through a rule base) is an excellent input to animations or test executions of different scenarios, to make them transparent even to stakeholders and non-IT roles. That story is worth another blog post, later this spring.

by Milan Kratochvil

Trainer at Informator, senior modeling and architecture consultant at Kiseldalen.com, main author : UML Extra Light (Cambridge University Press) and Growing Modular (Springer), Advanced UML2 Professional (OCUP cert level 3/3).Milan and Informator collaborate since 1996 on architecture, modelling, UML, requirements, and design. You can meet him at public Architecture courses in English or Swedish ( T1101, T1430) in April, May, and June, or Modeling courses in May ( T2715, T2716).

Läs mer