Intelligible Intelligence: Deep XAI still more R&D than toolbox

Most architectural tradeoffs are hard. So is the one between the accuracy of Deep Machine Learning (ML) and explainability/transparency of explainable AI (XAI). Therefore, DARPA’s initial XAI program is expected to run through 2021.

Few explainability mechanisms have been extensively tested on humans, but current R&D indicates at least some (hybrid) tech to come in a couple of years.

Several well-tried ML technologies work their way through (in steps) from a first random solution to satisfactory ones, and where possible, to an optimal one:

·         Genetic Algorithms, by creating additional generations

·         Forests, by creating additional trees from the same data

·         Deep Neural Networks, by a (cost) function applied (in training) to outputs, based on how they differ from labeled data, and propagated back across all neurons/synapses, to adjust weights

·         Hybrids, by combination (the “new kid on the block”).

NASA Space Technology 5 antenna created by GA (source: NASA.gov)

GA and Forests are more than semi-intelligible, by their nature. However, now that deep learning feeds big data through NN that consist of multiple hidden layers of neurons (DNN), it arrives at very accurate solutions to complex multidimensional problems, but at the same time at an inherent black box.

A part of my post from August is about random forests, which offer both more transparency than DNN do plus quite a degree of accuracy. Moreover, trees and NN are even cross-fertilized, to offer explainability without impeding accuracy, and there are already several flavors of this; to pick a handful:

·         adding extraction of simplified explainable models (e.g. trees) onto black-box DNN

·         local (instance-based) explanation of one use case at a time, with its input values, for example animating & explaining its path layer-by layer (like most test tools do).

·         soft trees, with NN-based leaf nodes, that perform better than trees induced directly from the same data

·         adaptive neural trees and deep neural decision forests that create trees (edges, splits, and leafs) , to outperform “standalone” NN as well as trees/forests that skip the combination.

 

Explainability, transparency, and V&V are absolutely essential to users’ reliance/confidence in mission-critical AI. Therefore, whichever path or paths take us to up-and-running products, XAI is welcome.

by Milan Kratochvil

Trainer at Informator, senior modeling and architecture consultant at Kiseldalen’s, main author: UML Extra Light (Cambridge University Press) and Growing Modular (Springer), Advanced UML2 Professional (OCUP cert level 3/3). Milan and Informator collaborate since 1996 on architecture, modelling, UML, requirements, rules/AI, and design. You can meet him at public courses (in English or Swedish) on AI, Architecture, and ML (T1913 , in December or February), Architecture (T1101, T1430) or Modeling (T2715, T2716).

Läs mer

Twin examples of multiple trees: 1. UML models, 2. Machine Learning

“Today, professionals get trained in using tools…  there’s a lack of education of fundamentals like modeling, architecture, methods, or concepts... Getting value out of data needs professionalization based on education and practical experience.”                                                                 

Andreas Buckenhofer, Daimler TSS, in an interview by OODBMS on Big Data, July 2019 

In my opinion, he’s spot on.

My post from March mentions why new AI languages aren’t exactly heavies of a CV in a mainstream business; in April, a figure (at the end of the post) also touched on Forest structures in ML and eXplainable AI.

After a free-wind sail that took us from detail to architecture, we now go into some structural “forestry”. It’s about tackling the same domain from multiple viewpoints, instead of clinging on to one.

1. Multiple trees in UML: Generalization sets

This post from 2015 (in Swedish) discusses the sets in more detail, so let’s just recap the diagrams, in English, and add «powertype» on the fourth one (a power set’s instances are subsets, so by the same token, a UML2 powertype’s instances are “subtypes” of a general construct).

2. Multiple trees in Machine Learning: random Decision Forests

Here, a designer is not the one who maps out subclasses. Rather, an ML algorithm generates in training time, from (labeled) data, an ability to perform classification (i.e accurate “mapping-out” of “classes”). The decision nodes of a (classification) tree gradually subdivide the data into more and more fine-grained classes.

Why bother about decision trees when Deep Neural Networks are booming? Because explainability opens the door to acceptance in mission-critical apps. ML-generated logic has to be auditable. User enterprises are pushing for graphicness, conceptualization, traceability, V&V. Those are the strengths of decision trees, and weaknesses of Deep NNs (we know those work, but hardly how); same thing with learning time required, size of training data sets, execution speed, or partitionability (a hint for IT architects: a tree works independently, whereas a neuron relies on many other ones). Atop of that, decision trees offer a structural backbone of hybrid AI systems (see also the last paragraphs in this post).

You might remember that trees in woods sometimes fuse their roots and exchange materials. Unsurprisingly, we find some synergy in virtual forests too. Firstly, our trees are “grown” on a random sample each (hence some “biodiversity” too), from one training-data set (hence fewer terabytes of training data). Secondly, on each sample, its tree’s decision nodes use a random subset of all available attributes. This gives architects and other roles some room to tune the mix of efficiency and explainability; in forests, it’s is near the level of genetic algorithms (GAs too have possible “mix-tuning points” in “biodiversity steps” Crossover and Mutation).

The more trees and “biodiversity” our ML algorithm grows, the more accurate and robust the generated logic becomes, because the final step is vote counting. A forest’s output (a classification like here, or a forecast) is an aggregated value of the outputs of all trees (a statistical mode in classification, or a mean in regression). It prevents the random decision forest from getting stuck in local optima, that is, we minimize error rates and overfitting to a given training-data set (which may be both incomplete and biased).

by Milan Kratochvil

Trainer at Informator, senior modeling and architecture consultant at Kiseldalens, main author : UML Extra Light (Cambridge University Press) and Growing Modular (Springer), Advanced UML2 Professional (OCUP cert level 3/3).

Milan and Informator collaborate since 1996 on architecture, modelling, UML, requirements, rules, and design. You can meet him in September at public courses (in English or Swedish) on AI, Architecture, and ML (T1913), Architecture (T1101, T1430) or Modeling (T2715, T2716).

Läs mer

6 Things AI and Machine Learning Reshape in SW

In my recent post, I mentioned that AI and ML challenge architecture, but also offer tools to tackle the challenges. Needless to say, automation of repetitive tasks will change our job descriptions, just like those of our end users.

The SW development lifecycle is changing too. Architects with experience from an environment with external content prosumers or open-source developers, will find some lifecycle changes similar to those two environments.

1. Crowd management

Much of the “crowd” involved is outside the IT organization (e.g. experts in domains other than IT), or even outside the enterprise - not least in Data as a Service offering ML from data sources such as digital twins (of customer-owned equipment, e.g. railroads and trains at Siemens, networks at Ericsson, or farming machinery at John Deere). Architects or CIOs have influence rather than full control, unlike in internal projects.

2. Crowdsourcing

Both the data and some ML-generated logic come from external sources, not least when some ML and computing runs locally on “edge” devices that produce the input data.

3. Adaptive planning

Distinct project phases tend to disappear, partly because of the “crowd” out there, partly because of the explorative nature of ML (“think more like a researcher, less like a programmer”). For example, a partial result of an ML project can hint about additional key domains to drill into, thus widening the scope and postponing the deadline.

4. Incomplete requirements

Customers may have a rather sketchy idea of what they want. “More bang for the buck” wouldn’t hint on “increased harvest, better soils, 80 percent less herbicides due to spraying individual weed plants only”, but ML with fast pattern recognition (in RT field images) does exactly that.

5. Widened job roles

Apart from the SW (development) lifecycle and the nature of new apps, AI reshapes the way they’re developed and thus our roles too. Architects and some devs become even curators of training data sets, co-analysts of ML results, and a guide for experts from non-IT domains, to enable them to apply ML in their specific tasks.

6. A hard core (platform)

The core (e.g. an automated data/ML platform) has to be secure, robust, modular, reliable (fault-tolerant, even on external error), documented and teachable to teams within the enterprise. The ML-generated system has to interoperate with other, programmer-made, systems (pre-ML AI, and other SW). The ML-generated logic has to be auditable and verifiable; indeed, explainability is the door to acceptance in mission-critical apps.

Figure from course AI, Architecture, and Machine Learning (T1913)

by Milan Kratochvil

Trainer at Informator, senior modeling and architecture consultant at Kiseldalen.com, main author : UML Extra Light (Cambridge University Press) and Growing Modular (Springer), Advanced UML2 Professional (OCUP cert level 3/3).

  


Milan and Informator collaborate since 1996 on architecture, modelling, UML, requirements, rules, and design. You can meet him this Spring at public courses (in English or Swedish) on AI, Architecture, and Machine Learning  (T1913),  Architecture (T1101, T1430) or Modeling  (T2715, T2716).

Läs mer

Yet another AI language you miss in your CV? 4 reasons why it will matter less and less.

It never hurts, but it varies how helpful a (fairly) new programming or script language is. From more or less a prerequisite in R&D and platform-vendor firms, to a nice-to-have CV footnote in mainstream businesses that rather emphasize extended SQL, analytics , data architecture, and automated ML platform/s.

Here are 4 reasonswhy you can live with it (ascending order by weight)

1. The fate of LISP, Prolog, Smalltalk, KQLM etc. (use comment form below to fill in what’s missing : ))

To drive the history of applied “AI 1.0” to the extremes, an enterprise in the 1980-ies was expected to achieve superpowers as soon as the CIO (and preferably, CEO : )) learned at least one exotic-enough AI language. The subsequent AI winter after 1990 happened supposedly because most CIO’s refused to so; best case: they got lost somewhere inside their fifth pair of nested parentheses in LISP (like most of us devs did too, including myself)…

2. The rise of expert-system development shells in business (near 1990)

Programming languages are versatile. You can get anything you need, given a generous timeframe. Development tools encapsulate a lot of technical detail. You can get more or less what you need, even under tight time constraints. No wonder it’s more appealing to CIOs than nested parentheses.

3. Success of those who used then-mainstream industry languages  

Books on configurators, or on LISP, drill down into Digital (HP) XCON/R1, but in books on management and modular manufacturing, you read about Scania Trucks & Buses (reporting profits for 80 consecutive years). Scania’s smart proprietary configurator (an age fellow of XCON) became a backbone of the enterprise, and grew to several times the size of XCON. Unlike XCON, Scania did cope in maintenance and upgrades. For decades. Offering complex customized vehicles assembled from a cluster of common component types. Language: unglamorous then-mainstream Cobol and DL/1.

4. A trend toward frameworks, component libraries, automated analytics & ML platforms

It’s a two-way street. On one hand, “AI 2.0” and ML challenge current architectures, not least data architectures: big-data ingestion, parallelism, fast access aid for non-sequential access because learning (in both human and artificial neural networks) is essentially non-sequential (parallel).

On the other hand, ML offers a toolbox to tackle these challenges, and also, enables quite some automation of the entire data pipeline and of an architect’s (or dev’s) repetitive tasks. I won’t be surprised if automated-ML platforms for big data, using extended SQL instead of script languages, spark success stories of Scania’s magnitude. It’s about the augmentation (or automation) itself, and about architecture fit for business, rather than about the detail.

So from now on, Informator’s new one-day course is called AI, Architecture, and Machine Learning. Neither just AI for Architecture, nor just Architecture for AI. It’s a two-way street.

Rapid progress in the middle

Figure from course AI, Architecture, and Machine Learning (T1913)

by Milan Kratochvil

Trainer at Informator, senior modeling and architecture consultant at Kiseldalen.com, main author : UML Extra Light (Cambridge University Press) and Growing Modular (Springer), Advanced UML2 Professional (OCUP cert level 3/3).

Milan and Informator collaborate since 1996 on architecture, modelling, UML, requirements, rules, and design. You can meet him this Spring at public Architecture courses in English or Swedish (T1913, T1101, T1430) or Modeling courses ( T2715, T2716)

     

Läs mer

Konsument-IT som liknar företags-IT

Christer Norströms presentation under SICS Open House 2018 handlade om hans intelligenta träningstjänst Racefox. Den stödjer individanpassad uthållighetsträning för löpare och längdåkare. Arkitekturen och kraven på Intelligenta träningsassistenter ligger ungefär halvvägs mellan ”vanlig” konsument-IT (gadgets) och företags-IT (mission critical).

Expertanvändare i gränslandet

Christer (tidare VD på SICS) med sina slides fick mig att mer eller mindre ångra två saker. Dels att jag la av med triathlon för ett antal år sedan. Dels att jag brukar dra den vanliga gränsen mellan B2C (Business-to-consumer) och B2B lite väl skarpt, oftast för att lyfta fram skillnaden i kvalitetskraven mellan dem.

Med tiden har Racefoxs logik utökats med skadeförebyggande eller –rehabiliterande inslag i träningen, i realtid, och då landar vi i gränslandet: expertkonsumenter minst lika pålästa om sin nischproblematik som processägaren på ett företag är om sin. Höga krav på tillförlitlighet, snabbhet (RT), exakthet. Samtidigt högt ställda förväntningar på att behålla kunderna (Racefox är uppe in en customer-retention faktor efter 1 år på 95%).

Arkitektur och arkitektroll
Autonoma Machine Learning (ML) system är ofta indelade i fyra lager, inte så olikt RT-system:
1. Perception (från input genom t ex Racefox kroppssensorer eller bilens givare)

2. Mönsterigenkänning (t ex kroppsställning och stavtag hos Racefox eller ojämn gång i en motor)
3. Analys, resonemang, beslut (dvs mönsterutvärdering och val av nästa steg)
4. Direkt åtgärd (t ex minska varvtal) eller interaktion (t ex varning genom genererad röst i skidlöparens hörsnäcka, eller ett anrop på bilverkstadens diagnostiksystem över Internet of Things).

Layered (arkitekturmönster). Modifierat från Informators kurs Avancerad Modellering (T2716).

Varje lager kan innehålla även ML-komponenter (t ex neurala nätverk tränade för sin nisch). Det gör en stor och trogen användarbas extra intressant, eftersom systemet blir smartare med tiden då det lär av allt längre tidsserier data från alltfler individer.

Bästa bakgrunden för en arkitekt blir då kunskap om domänen, om affären, om ML, om varje lager, och om samspelet mellan dem. Christer Norström kallade rollen value architect.

Milan Kratochvil
Informatorlärare, senior modeling & architecture consultant Kiseldalen.com, huvudförfattare: UML Extra Light (Cambridge University Press) and Growing Modular (Springer), Advanced UML2 Professional (OCUP cert level 3/3).

Milan och Informator samarbetar sedan 1996 inom arkitektur, modellering, UML, krav, och design. Du kan träffa honom på öppna kurser i Arkitektur på engelska eller svenska ( T1101, T1430) i maj och juni, eller i Modellering i maj ( T2715, T2716).

Läs mer

Leveled up: Auditability of AI and Machine Learning

Architects and many others remember the tightrope walk between flexibility/performance and testability/predictability/V&V in systems with many run-time parameters, or parallelism, or late binding time ranging from polymorphism to SOA-UDDI and ad-hoc computing. Now, it’s leveled up by ML (machine learning). Essentially the same tradeoff, but growing broader and trickier. 

Black box 

ML stirs up the fire; despite its roots (rule induction and mining) in the successful decryption of an unbreakable cipher, its near future looks encoded in weight values somewhere in deep neural networks. Whereas black-box flight recorders clarified the chain of events & decisions in past emergencies, more and more IT is now landing in black boxes that hide opaque logic.

Predictability wasn’t a big deal in consumer IT and entertainment (when Youtube or Spotify wrongly offered you a title you were avoiding like the plague, you rarely asked why)… If you just say “skis this wide apart look amusing”, I guess you’re in consumer IT, but if you insist on a layer-by-layer explanation why most artificial vision systems have a hard time in strong sunshine on white slopes, I bet you’re in corporate (image: Ski Robot Challenge, Korea). 

Tackle it one-way or two-way

Business apps are very different from apps for billions of consumers (see slides 9 to 15 in this talk by Oracle’s VP at SICS). Your enterprise or team can tackle the leveled-up tradeoff both top-down and bottom-up:

• assuring a framework of corporate values and procedures (particularly transparency, governance & compliance, accountability, and a security & safety culture)

• applying appropriate technologies and practices in IT to build in mechanisms upfront  for auditability, comprehensibility, predictability, traceability, testability/V&V (as well as fraud-prevention, such as restricted access to learning-data sets).       

On the latter (bottom-up) part, there’s ongoing AI research to “unpack” the opaque logic buried within deep learning systems, and to give them an ability to explain themselves. DARPA’s Explainable AI Program, XAI , aims at ML techniques (new or improved) that produce more explainable models, while maintaining a high level of prediction accuracy. New machine-learning systems will have the ability to explain their rationale, strengths, weaknesses, etc.

Hybrid-AI tech vendors often address organizations with more constrained schedules, budgets and levels of AI expertise. Hybrid learning systems combine “subsymbolic” ML with transparent symbolic computation (typically, wellknown knowledge-processing techniques). The combination lowers the total cost of entry into AI and ML, because it evolves from logic that domain experts already know (rules, decision trees, etc.)

From there, hybrid systems employ ML iteratively to fine-tune this explicit logic: for example, to narrow the IF-part of a rule to factors that prove most significant. That is, results of ML from big data decide about variables to be included (or omitted), about intervalization of a continuum of values, or about relevant threshold values of a particular variable.

Notably, a rule is still expressed as a rule yet with an ever-smarter and more accurate IF-part. This is transparent to humans, and paves the way to embedding AI and ML into daily IT-dev practice: devs and architects will gradually find thousands of decision points, enterprise-wide, suited for small AI apps in daily business. Those will generate valuable skills, know-how, and “tip feeling” as to where ML can work (or can’t).

Models, animations, transparency

Once the opaque logic is unpacked, or expressed as rules or trees, it’s time to revive your team’s modeling skills. Long story short, a decision tree (or an invocation path through a rule base) is an excellent input to animations or test executions of different scenarios, to make them transparent even to stakeholders and non-IT roles. That story is worth another blog post, later this spring.

by Milan Kratochvil

Trainer at Informator, senior modeling and architecture consultant at Kiseldalen.com, main author : UML Extra Light (Cambridge University Press) and Growing Modular (Springer), Advanced UML2 Professional (OCUP cert level 3/3).Milan and Informator collaborate since 1996 on architecture, modelling, UML, requirements, and design. You can meet him at public Architecture courses in English or Swedish ( T1101, T1430) in April, May, and June, or Modeling courses in May ( T2715, T2716).

Läs mer

Architects beware: 60 years since Dartmouth

Many R&D-intensive industries experienced an initial period of teething troubles, about six decades between their seminal events and their commercial breakthrough, followed by exponential growth. Last summer, 60 years had passed since the 1956 Dartmouth Artificial Intelligence Conference. 

History…

In 1887, Ernst Mach, a physics professor at the Charles University in Prague, established the principles of supersonics and the Mach number relating velocity to the velocity of sound (thus inspiring his faculty successor Albert Einstein’s theory of relativity). Exactly 60 years after, test pilot Chuck Yeager reached the magical speed of Mach 1, breaking the soundbarrier, with the Bell X-1 rocket plane.

From there, Mach numbers skyrocketed to NASA’s Apollo missions, taking humans to the Moon and back. In aerospace, “the sky is the limit” applied to turnover figures as well.

Mach (source: Wikipedia.org)                         Mach 1 (source: Wikipedia.org)

In 1865, the scientific community (including Charles Darwin) missed the importance of Gregor Mendel’s research in Brno into inheritance in plants, but rediscovered Mendel’s Laws in the 20^th century. Mendelian genetics and Darwin's natural selection finally merged in the 1930s, as evolutionary biology. Six decades later (1990-2003) the Human Genome Project HGP, the world's largest collaborative biological project so far, sequenced 92% of the human genome.

Genetics became a fast-grower with applications in diagnostics, forensics, archaeology, and more.

…repeats itself…

In 2016 (well, guess how many years after the Dartmouth AI conference) , the accuracy of Machine Learning (ML) systems started to outperform humans in extreme tasks, previously regarded as “out of reach” for AI. Some recent milestones are the games of Go and Poker , the latter by Mach’s and Einstein’s faculty heirs in Prague, and the University of Alberta Computer Poker Research Group in Edmonton.

AI delivers, which attracts brains and funds into the field. With the usual 60 years of teething in mind, we might call this the end of the beginning. AI departments of large US corporations in a variety of industry sectors are hiring AI experts by hundreds.

Yet the technical progress looks less dramatic when compared to the pace of both corporate and social change it catalyzes. A Forrester prediction last fall said 16% of US jobs will be lost to intelligent systems in the near future, and only partly compensated by 9% new jobs created by them (notably, jobs rather different from those that are vanishing).

…with an impact on architectural roles & landscape:

1. Much more IT(A) in Enterprise Architecture

EA will benefit from a stronger technical background. EA roles, architecture groups, and entire corporations who are used to absorbing new technology and have a strong background in IT including AI, have a competitive edge.

2. More tech leadership in management

That’s what built industries such as Scania, ABB, Volvo AB, and their modular configure-to-order tradition (C2O). The current shift in IT is more manageable in cultures with a clear context and clear ideas of what they need forefront tech for. After decades of custom-tailored complex manufacturing, people in these organization can come up with tangible proposals about leveraging for example, BI and CI (customer insight) downstream: in bidding, sales, pricing, assembly planning, flexible automation solutions, or within the product itself, e.g. in autonomous vehicles.

3. Robotics outcompete offshoring

I argued ten years ago that robots and automation offered a more long-term profitable solution. Everybody continued to rush offshore anyway, although the underlying figures weren’t convincing. Now (guess how many years after the Dartmouth AI conference… ) , AI has triggered a U-turn in corporate sentiment. By 2018, the number of manufacturing jobs moving from Sweden is going to equal the number of jobs moving back. The driving force: robotics and automation.

4. Architecture business as usual…

Architects often work with fancy tech within nearly medieval organizations under nearly stone-age governments. AI 2.0 might therefore feel painstaking. Intelligent robots can result in perpetual reorganizations (process innovators Michael Hammer and B. E. Willoch likened them to reshuffling the deck chairs aboard Titanic), and governments in high-tech countries, socialist and conservative alike, can spend billions on “creating very simple jobs” which is like herding cats: the simpler the jobs, the faster they jump (offshore, as some Swedish trade-union economists point out). Not to mention creating not-so-simple robot taxes that can push offshore the industries of an entire country or continent.

Architects aren’t enthusiastic about the mismatch they had to live with for a long time: a surplus of complexity and information, but a shortage of cognition; in data as well as in society…

5. New flavors of Architecture Patterns

For example, the Layered pattern, typical of business systems (UI, business logic, Object-Relational mapping, and DB) has siblings in deep-learning systems with layers of artificial neural networks trained for a key task each: perception (input parsing), pattern recognition, reasoning (pattern classification and selection of steps to take), and either autonomous action (“vehicle brakes on”, for example) or interaction (e.g. voice generation, or calls to other systems).

6. Ever-bigger data versus custom-fit learning strategies

Accurate fast learning from small data has an architectural savings potential, rarely mentioned in the big-data buzz. Two routes can take you there:

a)  pre-trained neural networks off the shelf (nowadays, you find those even in Matlab) to solve a certain category of problems, and ready to be extra-trained just for the “delta” i.e. the specifics of yours. Largely 90+ percent of the precision, at a fraction of the training time and cost.

b) cross-breeds of several AI techniques, as indicated by Poker systems where an innovative adaptation of a well-proven algorithm made DeepStack run quite fast on a laptop, no longer requiring extreme searches running on supercomputers.

7. Auditability, comprehensibility, V&V, reviews by humans

This category of ML challenges would be worth an entire blogsite. The tradeoff between quality (accuracy of output) and auditability (comprehensibility of machine-made internal logic) grew trickier generation by generation of ML technologies.

To cut a long story short, it’s easier to test that the “sub-symbolic” logic works accurately, than to see why or how.

   

Summing up

Neither Enterprise nor IT Architecture is exempt from AI’s impact on business processes and technology. Machine learning affects systems, organizations, and society, from the way an architect can tweak a plain pattern, and up to the way policymakers can get things plain wrong…

by Milan Kratochvil

Trainer at Informator, senior modeling and architecture consultant at Kiseldalen.com,

main author : UML Extra Light (Cambridge University Press) and Growing Modular (Springer), Advanced UML2 Professional (OCUP cert level 3/3).

Milan and Informator collaborate since 1996 on architecture, modelling, UML, requirements, analysis and design. In the next couple of months, you can meet him at Architecture ( T1101 , T1430, in English or Swedish) or Modeling courses ( T2715, T2716 , mostly in Swedish).

x

Läs mer