Animators, Auditability, AI: high-tech history and a near future

When the audience doesn’t get your message, why not “send the text to Hollywood”… An idea almost as old as engineering itself. Even in the past, movie components boosted transparency and accelerated the mass adoption of new technologies, or logic, that had seemed too black-box before.



1929

E. A. Link launched the first commercially available flight simulators, called the LinkTrainers. They became efficient tools to teach flight and to show the behavioral “logic” of an airplane to novices and junior pilots, in a safe setup (on the ground). By WWII, Link delivered over 10,000 and they became standard equipment at every air training school, for pilots from all Allied nations.     



1960’s

E. W. Dijkstra laid the foundation of structured programming, software architecture, and algorithm visualization. Also, he pointed out the limited ability of (static) lines of code to make (dynamic) behavior clear, and he realized the expressive power of visualizing it dynamically; put together, this paved the road to animators.



1980’s

Algorithm-independent animators started to emerge as lab tools and research papers. Most of them animated source code, highlighting it line by line during test execution. In 1988, I worked with senior consultant Leif Carlson of Jackson Systems International on the integration of the first PC-based code animator off-the-shelf (by Micro Focus UK) with a Swedish modeling tool and code generator, the JSP-Tool. The AnimatorLink made it possible to animate behavior both as diagrams and as lines of generated code, in sync, on one screen. It improved design transparency as well as testability.



1990’s

Animators rolled on, into Business Process Modeling as well as into technical documentation in manufacturing where they complemented CAD and knowledge-based tools, especially in automotive electronics and in other sectors where electronics augmented many (mostly mechanical) components. Animations turned out excellent to teach senior staff, such as mechanics or drivers, “why a modern engine works at all”.



1997

UML 1.0 was adopted as a standard, which channeled more animation effort into UML modeling. Among the first off-the-shelf animators using UML were Rational/RT, and Select Business Solutions UK (I often demonstrated Select’s Object Animator at my public UML courses, to underline behavior in the pre-SOA era).



2005

UML 2 gradually became a standard in 2003-2005. TeleLogic in Lund, Sweden, offered animation of UML2 Sequence Diagrams, states, and code, all in one partitioned window. Architects, testers, and devs could use a Sequence Diagram animator as an interactive tester GUI, among other things. A beta was available already a year or two before UML 2 took its final shape.





2010

Some organizations I’ve advised on architecture and modeling over the past 5-10 years leverage user-story or use-case animations to communicate their architecture (ITA, EA, SA, or even legacy system portfolios) to a variety of roles and backgrounds. If you’re in a primarily-tech business, you can animate some of the UML (or SysML) behavior diagrams, off-the-shelf. Whereas, if you’re in finance, government & administration, or care, with many non-technical roles, then it’s best to go for one type of diagram throughout (which means, even in animations).

Often, a layered UML Component Diagram (sometimes nicked “architect’s diagram”) is agreed upon; however, most UML modeling tools animate behavior diagrams, so with other diagram types (i.e. with structure) it’s practical to hire someone with a very versatile animator tool to “project” your scenarios directly onto your structure. History tends to repeat itself. In navigator training, some simulators for WWII flight schools could play a sequence of ground photographs (e.g. an approach route), like a movie in the navigator’s “window”. The instructor kept, manually, the “ground movie” in pace with the speed indications on dashboard; to the trainee, it looked very similar to flying across the terrain.        



After today’s leap in Machine Learning

In my previous blogpost, I mentioned that appropriate technologies, practices, and built-in mechanisms for auditability, comprehensibility, predictability, traceability, and testability/V&V are top of the AI agenda.       

As soon as the opaque logic generated in deep neural networks (in their training phase) can either be unpacked afterwards, or structured upfront during training (into rules or trees or case bases), it’s high time to animate or trace various scenarios through that logic. Just like in animations in the past, this will clarify things to stakeholders, non-IT roles, trainees, or new employees.




Trainer at Informator, senior modeling and architecture consultant at Kiseldalen.com, main author : UML Extra Light (Cambridge University Press) and Growing Modular (Springer), Advanced UML2 Professional (OCUP cert level 3/3).

 

Milan and Informator collaborate since 1996 on architecture, modelling, UML, requirements, and design. You can meet him at public Architecture courses in English or Swedish T1101, T1430) in April, May, and June, or Modeling courses in May ( T2715T2716)


Varför TypeScript?

Frontend-utveckling i JavaScript har alltid varit full av fallgropar. Ett moget team känner till dessa och tar höjd för dem — med linters, tester, best practices, style guides, etc. Alla dessa saker höjer kodkvalitén, men de adresserar inte det grundläggande problemet: att JavaScript som språk är illa rustat för utveckling av större applikationer.

TypeScript löser det problemet. Från dag 1 med TypeScript fångar man misstag som JavaScript glatt struntar i. Man kan bygga större och mer ambitiöst, och få bättre kontroll över applikationens arkitektur. Samtidigt har man med sig och kan återanvända sin kunskap om JavaScript.

I JavaScript är det OK att glömma en property i en objektliteral, att anropa en funktion med färre eller fler argument än vad det var tänkt, eller att skriva kod som aldrig kommer att nås i ett program. TypeScript flaggar upp alla dessa saker som fel, oftast direkt i ens IDE.

Läs mer om vår TypeScript-kurs.

TEDx Talks: Mia Liljeberg - Changing the way we change


Mia Liljeberg works with leaders to upgrade them to survive and thrive in the VUCA world. Building resilient leaders and calibrating their organizations to stay ahead of the game. With over 15 years of change management in the luggage, and having lived and worked in over 14 countries on five continents for global companies in most industries makes it easy for her to chart and navigate in new environments. Do you know what corporate culture you are cultivating? Do you get the best of your employees in a culture of shared vision, trust and co-operation where they are being good and unique results are created? Mia Liljeberg is the side-kick during organizational changes, big and small. With over 15 years of international change management in the luggage, she shows, guides, coaches organizations throughout the process in a simple and engaging way. Engineering background and have lived and worked in over 14 countries on five continents in most industries makes it easy for her to chart and navigate in new environments.

Attend one of Mia's training courses this spring!

Are you swimming upstream and getting exhausted at work?

In many organisations people work very hard, but due to lack of clear direction or focus on key priorities, many overwork. They try harder and harder to swim upstream, but don’t get to the expected results.
In the risk zone are for example those people, who have very strong sense of Honour motive. Those with high Honour are very loyal towards their employer (even though the employer might not be as loyal towards them). They want to keep their promises and will do their best to try to finish all the work at hand. But often there is not enough of time to do it all well, and with lack of overall direction or priorities, they risk getting burned out.
This might be difficult to understand, if you yourself don’t have the same high motive. But with the analogy of some people choosing the military career, to be prepared even to die for their country, you might understand how different we people are.
Honour is one of the 16 motives all people have, more or less.
If you would like to know your motivation profile and learn more from a scientifically validated method: Informator is conducting trainings in Reiss Motivation Profile:
These trainings can also be tailored for teams (in English, Swedish and Finnish).
Aila Kekkonen
Better Future

Personnel and Leadership Development

Leveled up: Auditability of AI and Machine Learning


Architects and many others remember the tightrope walk between flexibility/performance and testability/predictability/V&V in systems with many run-time parameters, or parallelism, or late binding time ranging from polymorphism to SOA-UDDI and ad-hoc computing. Now, it’s leveled up by ML (machine learning). Essentially the same tradeoff, but growing broader and trickier. 

Black box 

ML stirs up the fire; despite its roots (rule induction and mining) in the successful decryption of an unbreakable cipher, its near future looks encoded in weight values somewhere in deep neural networks. Whereas black-box flight recorders clarified the chain of events & decisions in past emergencies, more and more IT is now landing in black boxes that hide opaque logic.

Predictability wasn’t a big deal in consumer IT and entertainment (when Youtube or Spotify wrongly offered you a title you were avoiding like the plague, you rarely asked why)… If you just say “skis this wide apart look amusing”, I guess you’re in consumer IT, but if you insist on a layer-by-layer explanation why most artificial vision systems have a hard time in strong sunshine on white slopes, I bet you’re in corporate (image: Ski Robot Challenge, Korea). 

Tackle it one-way or two-way

Business apps are very different from apps for billions of consumers (see slides 9 to 15 in this talk by Oracle’s VP at SICS). Your enterprise or team can tackle the leveled-up tradeoff both top-down and bottom-up:

  • assuring a framework of corporate values and procedures (particularly transparency, governance & compliance, accountability, and a security & safety culture)
  • applying appropriate technologies and practices in IT to build in mechanisms upfront  for auditability, comprehensibility, predictability, traceability, testability/V&V (as well as fraud-prevention, such as restricted access to learning-data sets).       

On the latter (bottom-up) part, there’s ongoing AI research to “unpack” the opaque logic buried within deep learning systems, and to give them an ability to explain themselves. DARPA’s Explainable AI Program, XAI , aims at ML techniques (new or improved) that produce more explainable models, while maintaining a high level of prediction accuracy. New machine-learning systems will have the ability to explain their rationale, strengths, weaknesses, etc.

Hybrid-AI tech vendors often address organizations with more constrained schedules, budgets and levels of AI expertise. Hybrid learning systems combine “subsymbolic” ML with transparent symbolic computation (typically, wellknown knowledge-processing techniques). The combination lowers the total cost of entry into AI and ML, because it evolves from logic that domain experts already know (rules, decision trees, etc.)

From there, hybrid systems employ ML iteratively to fine-tune this explicit logic: for example, to narrow the IF-part of a rule to factors that prove most significant. That is, results of ML from big data decide about variables to be included (or omitted), about intervalization of a continuum of values, or about relevant threshold values of a particular variable.

Notably, a rule is still expressed as a rule yet with an ever-smarter and more accurate IF-part. This is transparent to humans, and paves the way to embedding AI and ML into daily IT-dev practice: devs and architects will gradually find thousands of decision points, enterprise-wide, suited for small AI apps in daily business. Those will generate valuable skills, know-how, and “tip feeling” as to where ML can work (or can’t).

Models, animations, transparency

Once the opaque logic is unpacked, or expressed as rules or trees, it’s time to revive your team’s modeling skills. Long story short, a decision tree (or an invocation path through a rule base) is an excellent input to animations or test executions of different scenarios, to make them transparent even to stakeholders and non-IT roles. That story is worth another blog post, later this spring.


by Milan Kratochvil
Trainer at Informator, senior modeling and architecture consultant at Kiseldalen.com, main author : UML Extra Light (Cambridge University Press) and Growing Modular (Springer), Advanced UML2 Professional (OCUP cert level 3/3).Milan and Informator collaborate since 1996 on architecture, modelling, UML, requirements, and design. You can meet him at public Architecture courses in English or Swedish ( T1101T1430) in April, May, and June, or Modeling courses in May ( T2715T2716).



CSS selectors are evil and JS is the solution

This blogpost is excellent as preparation for the course Javascript Foundation, running March 22-23.

The premise

In this blog post we'll demonstrate...
  • why CSS inherently doesn't scale
  • how BEM is a partial solution
  • how inline styles through JS is a (nearly) full solution
Towards the end we'll use React as a vehicle for our example, but the concepts apply equally well to Angular or any other modern JS framework. We won't use any "CSSinJS library".
A small caveat: according to many, what we're describing in this post isn't strictly CSSinJS, but inline styles via JS. However at this stage the distinction doesn't matter much, so if you came here because you're curious about CSSinJS, by all means read on!

Our example

Imagine we have a complex webapp with lots of styling. We now want to add a new UI concept; an explanation box, meant to be shown inside a bigger context to explain some related concept.
Here's what it should look like:
cssinjsbox.png
Disregarding styling, here's the markup:
<div>

  <h2>Did you know?</h2>

  <p>Ozelots hate maccaws!</p>

</div>
We want to apply these styles to the explanation box div:
border-radius: 15px;

padding: 10px;

border: 2px solid springgreen;

background-color: moccasin;

color: magenta;
And inside the box we want these styles for the explanation headline h2:
font-weight: normal;

letter-spacing: 5px;

padding-left: 15px;

font-style: italic;

font-size: 14px;
As an added spice, also imagine these styles being applied to every h2 in our app:
font-family: Verdana;

font-size: 16px;
Now let's start exploring how to add these styles to our markup!

Take 0 - inline styles

The most primitive solution is to simply shove all needed style rules into the style prop of every explanation box:
<div style="border-radius: 15px; padding: 10px; border: 2px solid springgreen; background-color: moccasin; color: magenta;">

    <h2 style="font-family: Verdana; font-size: 14px; font-weight: normal; letter-spacing: 5px; padding-left: 15px; font-style: italic;">Did you know?</h2>

    <p>Ozelots hate maccaws!</p>

  </div>
This is of course a horrible solution since
  • the HTML becomes super-bloated and unreadable
  • the styles can't be reused
  • we'd be repeating "theme rules" (like the moccasin background color) all over the place

Take 1 - stylesheet with selectors

To counter the above we always put our styles in a separate CSS file, using selectors to direct where they are applied.
In order to be able to target explanation boxes we can introduce a class name to the markup:
<div class="explanationBox">

  <h2>Did you know?</h2>

  <p>Ozelots hate maccaws!</p>

</div>
In a separate CSS file we can now apply the box styles like this:
.explanationBox {

  border-radius: 15px;

  padding: 10px;

  border: 2px solid springgreen;

  background-color: moccasin;

  color: magenta;

}
We style the explanation headline by using a child combinator in the selector:
.explanationBox > h2 {

  font-weight: normal;

  letter-spacing: 5px;

  padding-left: 15px;

  font-style: italic;

  font-size: 14px;

}
The generic h2 styles would get their own selector:
h2 {

  font-family: Verdana;

  font-size: 16px;

}
Note that both of these later selectors will have an opinion about the font-size of our explanation headline, but since .explanationBox > h2 is more specific than h2, the former will win.

Unanswerable question 1 - the unknown killer problem

With our styles in a CSS file, we now have the following situation:
cssinjs-selector.svg
The rule blobs are connected with their respective target elements via selectors. While they work fine for our simple example, they don't scale well. As the app grows it is very easy to end up in a messy situation where we are scared to change both HTML and CSS, in fear of some selector somewhere breaking.
Even in our small example the main reasons for the unscalability are easy to demonstrate.
Look again at our HTML...
<div class="explanationBox">

  <h2>Did you know?</h2>

  <p>Ozelots hate maccaws!</p>

</div>
...and ponder this simple question with regards to the div:
What styles are applied to this element?
We have no easy way of knowing! The elements are being shot at by CSS selectors somewhere out there in the dark. From this side we have no idea whether any of them are targetting this specific element.
It is likely that there is a selector targetting the .explanationBox class in a selector somewhere. But because CSS selectors are global we'd have to search all CSS in order to know.
And consider asking the same question about the h2 - it doesn't have a class, which might imply it is not in the crosshairs, but it is in fact targeted by two selectors; h2 and .explanationBox > h2.
Just by looking at the HTML we cannot know what styles, if any, are applied to an element.

Unanswerable question 2 - the unknown victim problem

We find the reverse of the same problem in the CSS file. Look for example at this part:
.explanationBox > h2 {

  font-weight: normal;

  letter-spacing: 5px;

  padding-left: 15px;

  font-style: italic;

  font-size: 14px;

}
Now imagine trying to answer this question:
To which elements are these styles applied?
This is impossible to know. We'd have to search through all our HTML files, or templates in case of a dynamic app. And in a dynamic setting the .explanationBox element and the contained h2 might live in completely different places, making it even harder to figure out the answer.
This makes editing a CSS file really scary. Can we remove this part? If the question above was easy to answer then we could know whether it is safe to remove a certain part, but now we cannot!

Take 2 - using the BEM naming convention

There has been many attempts to mitigate the mess caused by the inability to answer these two questions. One such attempt is the BEM naming convention, which basically boils down to...
  • have selectors consist of a single class always
  • name the classes according to a strict convention
For our example, the explanation box would be a Block, while the h2 inside of that would be an Element of that block. This is the B and E of BEM (the M stands for modifier but we don't need that here).
Using BEM logic our markup would become:
<div class="explanationBox">

  <h2 class="explanationBox__headline">Did you know?</h2>

  <p>Ozelots hate maccaws!</p>

</div>
The CSS would now be:
h2 {

  font-family: Verdana;

  font-size: 14px;

}



.explanationBox {

  border-radius: 15px;

  padding: 10px;

  border: 2px solid springgreen;

  background-color: moccasin;

  color: magenta;

}



.explanationBox__headline {

  font-weight: normal;

  letter-spacing: 5px;

  padding-left: 15px;

  font-style: italic;

}
It is now slightly more clear what styles are applied where. Both the earlier questions are easier to answer, since we can do a search for the more precise class names.
But we still have the problem of the generic h2 selector. Also class names might be dynamically created and/or added to an element, so again, in a dynamic setting it might still be hard to find out exactly what's going on.

Take 3 - inline styles in a React component

The basic idea in BEM, as well as in OOCSS or any of the other attempts at bringing structure to CSS, is to think in terms of components. All style definitions, as far as possible, should be tied to a certain component.
This jives very well with modern JS frameworks, who (almost) all have a component-centric architecture. If components are our main abstraction, then it is very easy to make an API to style individual components.
React (and most of the competition) does this by letting us provide a CSS-like JS object to the style prop:
const box = {

  borderRadius: "15px",

  padding: "10px",

  border: '2px solid springgreen',

  backgroundColor: 'moccasin',

  color: 'magenta'

};



const headline = {

  fontFamily: 'Verdana', // should live in generic h2 styling, but we're cheating for now

  fontWeight: "normal",

  letterSpacing: "5px",

  paddingLeft: "15px",

  fontStyle: "italic",

  fontSize: '14px'

};



export class ExplanationBox extends React.Component {

  render() {

    return (

      <div style={box}>

        <h2 style={headline}>{this.props.title}</h2>

        {this.props.children}

      </div>

    );

  }

}
Elsewhere in our React app we can now do this...
<ExplanationBox title="Did you know?">

  <p>Ozelots hate maccaws!</p>

</ExplanationBox>
...and we have our reusable, styled component!

Take 4 - imported inline styles

However, what you just saw was really nothing but a regression to our Take 0 attempt, where we put all styles inline. Like an animal.
But we can improve on this super easily, simply by migrating the styles to a separate file! Imagine the following living in a file named styles.js:
export const typography = {

  h2: {

    fontFamily: 'Verdana',

    fontSize: '16px'

  }

};



export const explanation = {

  box: {

    borderRadius: "15px",

    padding: "10px",

    border: '2px solid springgreen',

    backgroundColor: 'moccasin,

    color: 'magenta',

  },

  headline: {

    ...typography.h2,

    fontWeight: "normal",

    letterSpacing: "5px",

    paddingLeft: "15px",

    fontStyle: "italic",

    fontSize: '14px'

  }

};
Note the handling of the basic h2 styles!
Our React component now becomes this:
import {explanation} from styles;



export class ExplanationBox extends React.Component {

  render() {

    return (

      <div style={explanation.box}>

        <h2 style={explanation.headline}>{this.props.title}</h2>

        {this.props.children}

      </div>

    );

  }

}
We have now sort of reinvented the stylesheet! Except, where before the stylesheet used selectors to fire blindly into the dark...

cssinjs-selector.svg
...we now let the components go get exactly the styles they need via imports:

cssinjs-import.svg
It might not be immediately obvious, but the flipping of this arrow makes a world of difference!

Finding the killer

Remember the first unanswerable question?
What styles are applied to this element?
In our imported inline style setup, this becomes trivial to answer. I simply follow the thread backwards from the style prop!

cssinjs-findkiller.gif
Finding the victim
Our second unanswerable question was:
To which elements are these styles applied?
Again, because of the explicit imports, this becomes very easy to answer (if you have a semi-competent editor):

cssinjs-findvictim.gif
LESS isn't more
Through moving our style definitions to JS land, the need for CSS preprocessors goes out the window. Remember how we used the ... spread operator to mix in the h2 styles into our headline rules?
headline: {

  ...typography.h2,

  fontWeight: "normal",

  letterSpacing: "5px",

  paddingLeft: "15px",

  fontStyle: "italic",

  fontSize: '14px'

}
That was the equivalent of a LESS/SASS mixin! You'll quickly find that the other syntax they give us can be easily replicated with pure JS. For example, having global variables becomes trivial:
export const palette = {

  softcolor: 'moccasin',

  hardcolor: 'magenta',

  accentuationcolor: 'springgreen'

}



export const explanationBlock = {

  box: {

    borderRadius: "15px",

    padding: "10px",

    border: '2px solid ' + palette.accentuationcolor,

    backgroundColor: palette.softcolor,

    color: palette.hardcolor,

  },

  // ...truncated
As do implementing helper functions to add vendor prefixes, saturate colours, etc.

Wrapping up

To recap; by turning the arrows around through using JS and inline styles, almost all opaqueness of regular CSS goes away. Previously difficult tasks such as...
  • renaming a CSS class
  • identifying and removing dead CSS code
  • hitting the correct elements with selectors
  • NOT hitting the wrong elements with selectors
...suddenly become easy or entirely moot.
While there are still a lot of questions to be answered regarding moving CSS to JS space, there's no doubt in my mind that just being able to flip the arrow of responsibility makes it all worth it.
One of the primary downsides of inline JS styles is that we can't use pseudoselectors such as :hover or @media. I hope to show how we can get around that using helper libraries in an upcoming post.
--
David Waller, Edument
Edument AB was founded in 2010 by Acke Salem and Tore Nestenius. They met for the first time at a conference where they began discussing the lack of services in the development sector that included both education and mentoring. Hence the company name stems from EDUcation and MENTorship.

Today Edument has 25 employees and offices in Helsingborg, Malmö, Falkenberg and Prague. We have educated and mentoring thousands of developers around the world. And been awarded "Årets Gasell" by Dagens Industri two years in a row.