Archives for posts with tag: quality

The best way to explain MDE to someone without any model-driven experience is to involve in an MDE development. The second best, for which blog is a suitable medium, is to show examples. Today I would like to share an example of a simple MDE application and give a glimpse into work items and a process behind its development.

The application in question is an MDE implementation of a sequencing system described in [1]. The purpose of this MDE exercise was to quickly build something concrete that would help Luminis learn a new vertical domain.

Problem Domain and Demo System

Curriculum content sequencing (CCS) is an important pedagogical service. The purpose of this service is management of learning routes to help students achieve curriculum goals. An emerging trend in education is adaptive learning that is tailored to backgrounds and preferences of individual students. This is in contrast to the traditional way of content sequencing that usually prescribed a single learning route for groups of individuals. Advances in e-learning provide an excellent platform for adaptive content sequencing.

Actors_and_services

Figure 1: Use case diagram of the demo system

A use case diagram in Figure 1 shows actors and important services of a simple content sequencing system. These services are:

Capture sequencing content provides Sequencing Expert with tools to design content sequencing knowledge necessary to reach a curriculum goal. 

Register learning units enables Materials Provider to connect leaning materials to nodes of content sequencing.

Suggest a learning route provides Student with a learning route tailored to the student’s curriculum competences, background and preferences. The service also suggests necessary learning units and their substitutions from other providers.

Problem Domain Ontology

Domain ontology is conceptualization of domain knowledge. Figure 2 shows an information model that captures ontology for the CCS domain. The model is created using the Object Role Modelling (ORM) methodology. Some readers may be also familiar with ORM’s close relatives from The Netherlands: NIAM and FCO-IM.

Simple_ssc_ontology

Figure 2: Ontology for the curriculum content sequencing domain

At this point a few disclaimers are due:

The model loosely conforms to ORM, namely main principles are followed, used notation is based on more popular UML and ER languages, and not all ORM tools are being employed (an example of an ORM tool in Figure 2 is relationship ‘succession’ that is objectified as an entity).

The model itself is not complete. Moreover, it cannot be claimed correct, as the model was not verified with subject matter experts. However, the model is sufficiently accurate for the purpose of the demo (see above).

Design

The system illustrated in Figure 1 is partitioned in 2 parts: a domain specific modeling environment (DSME) for Sequencing Expert and a web-based CMS for Student and Materials Provider. Both parts can import/export curriculum content sequencing designs from each other. Each part provides the actors with rich environment and enables reuse of generic services that are relevant to the application, e.g. user management, document flow, security, content search. Any application-specific functionality that changes frequently during the application’s life-cycle was engineered with models, otherwise programmed. Such points of frequent change are known as variation points.

Figure 3 shows the system parts, variation points (shown as boxes) and activities (diamonds) related to changes in the variation points. These are laid over the OMG’s four modeling layers (labeled M0-M3). Variations at level M1 are part of the normal application operation. Here Sequencing Expert and Materials Publisher define sequencing designs and learning units (LU) respectively. Variations at level M2 occur when domain definitions (see Figure 2) change, e.g. as developer’s understanding of the CCS domain evolves. These are changes to the system itself and are carried out by developers during development or consequent system updates. Grayed out shapes at M0 and M3 are system and technology related variation points that do not change once chosen.

Ccs_meta_application_pim

Figure 3: System parts, modeling architecture, variation points and change processes

Changes at level M2 are expensive and (given the experimental nature of the demo) frequent. Therefore these are best reduced or avoided. The direction of development activities implies that there is a single source of M2 changes, from which all the others can be derived. MDE is applied to isolate this source of change within a model (see Metamodel in the figure) and automate related development activities by means of transformations.

The following are selected technologies:

  • AToM3 ─ a language workbench. AToM3 uses Entity-Relationship (ER) and Graph Grammar (GG) as Metalanguage and Transformation definition language respectively. AToM3 is used as language workbench in (model-driven) development and as domain-specific modeling environment (DMSE) for Sequencing Expert.
  • Zope ─ a web application server that is typically used as intranet and extranet server, document publishing system, portal server and platform for collaboration. In this application, Zope provides a web-based CMS for Student and Material Provider.
  • ZCase ─ a model-driven software factory for code generation of Zope document types.
  • Python for development other than model-driven.

With these solutions and technologies in place, the following is left to be developed:

  1. Metamodels for sequencing designs and learning units (see Figure 2).
  2. A transformation bridging the gap between AToM3 metamodels and ZCase, thus forming a complete transformation chain Model→Code.
  3. Import/export routine between AToM3 and Zope
  4. Simple web-interfaces and document search for Student and Materials Provider

The last two items were trivial to implement. Item 2, while simple, requires introduction of new material (namely ER, Class diagrams, Zope, ZCase, Python) for explanation. For these reasons items 2,3 and 4 will not be covered in the article.

Capturing Sequencing Designs

With the selected technology we can now define a DSL for capturing sequencing designs in AToM3.

Sequencing_dsl_metamodel

Figure 4: CCS metamodel

Figure 4 shows a CCS metamodel written in ER. Note, that the screenshot shows only the abstract syntax, but the other aspects (see metamodelling quality) are defined as well. The quality of the metamodel is at level 5.

Using language workbench capabilities and the above metamodel, AToM3 configures itself into a dedicated modeling environment for curriculum expert. (This configuration is implemented by AToM3’s own transformation Generate DSME.)

A_learning_sequence_example_of_the_first_grade_mathematics_course_determined_by_curriculum_experts

Figure 5: A learning sequencing design for the first grade mathematics course

Figure 5 is a screenshot of the CCS modeling environment at work: Left vertical toolbar contains buttons for every modeling concept defined in the CCS metamodel. Top horizontal toolbar contains generic modeling tools, Edit and Connect in particular. In AToM3 modeling is performed by means of a visual editor: one selects a modeling concept, places it on the canvas and modifies its properties with the Edit tool. Further on, such created instances of modeling concepts can be coupled using the Connect tool. An example of such a visual model is a sequencing design for the first grade mathematics course, shown on the canvas of DSME.

Sequencing Designs in CMS

Given the CCS metamodel, transformation chain ER→Zope generates implementation of the corresponding document types for Zope. Figure 6 shows these document types as seen in CMS (see options in the drop down list). You may recognize the entities and relationships from the metamodel shown in Figure 4.

Cms_sequencing_document_types

Figure 6: Sequencing document types available in CMS

These types allow curriculum experts to build sequencing designs in a web browser. A more convenient way is to transfer sequencing designs from the DSME part. This is achieved by executing an export transformation on a sequencing design in AToM3. The result of this transformation is a fully searchable CCS document stored as graph structure in Zope. For example, given the AToM3 model in Figure 5, export would create a Zope document of type Sequencing multigraph as shown in the figure below.

Author_view_on_a_sequencing_model

Figure 7: Sequencing expert view on a CCS document

Figure 7 shows the web-interface that represents a CCS document as graph structure, i.e. a set of edges and nodes. (A viewer which can display graph-like structures in HTML web pages would be more appropriate, but is not necessary for this demo.) Note that the structure and properties of the AToM3 model from Figure 5 are transferred without loss of information.

The document shown in Figure 7 can be changed online and downloaded (via export tab seen in the screenshot) as an AToM3 model. Consequently, the downloaded file can be loaded in AToM3 for editing, thus closing the import/export round-trip.

Learning Unit Registration

Learning_unit_model

Figure 8: Learning Unit metamodel

Learning Unit is a simple and atomic Zope document type, whose AToM3 metamodel is shown in Figure 8. LU Zope type is generated and its instances are created, searched, read, and modified similar to those of the CCS document types. A simple web interface is sufficient for Material Provider. 

Obtaining Learning Routes

Currently, this service has a very basic recommendation mechanism:

  • Sequencing designs are searched to match query parameters. 
  • No account is taken of student’s curriculum competences, background and preferences (and there is no student profile). 

Searching_for_course_sequencingReader_view_on_a_sequencing_modelSubstitutive_materials_per_sd

The above screenshots illustrate the current functionality: The first is the web interface that allows students to define search parameters and displays search results. The next is the web view of the sequencing design previously shown in Figure 5. Selecting any node of the design, will display information about the sequencing node and suggest learning units and their substitutions from alternative materials providers (see the last screenshot).

Summary

In conclusion, I would like to highlight a few points about this MDE application:

The complete development of the application took about 2 man-week. The real power, however is how fast the application can be updated as the knowledge of the application domain evolves: a matter of an hour given even drastic changes in metamodels. This efficiency is possible due to application of the SSoT principle, proper abstractions and automation: Effectively, two separate PSM developments were replaced by a single PIM development, from which code is generated for two platforms (AToM3 and Zope).

In development of this application, off-the-shelf MDE frameworks were reused (namely AToM3 and ZCase) and traditional development process was interwoven with development of missing model-driven assets (metamodels, ER→ZCase transformation). The latter is a software development process too, and as a matter of fact, followed OMG’s CIM/PIM/PSM process architecture.

Naturally, development of model-driven assets has its particularities as well. For one, there is a much stronger isomorphism between objects in the world of application users and domain concepts in model-driven artifacts. Therefore, role of analysis models, such as shown in Figure 2 is more important than in the more requirements-oriented traditional development.

This demo provides extremely simplified service for learning route recommendation. Recommendation algorithm depends on the information structures and is sensitive to changes in those structures as well. The service is in fact a model interpretation, where model is a CCS design and a Student profile. A possible MDE approach to building an application-specific interpreter is described here (but be sure to read the approach’s limitations as well).

Developed metamodels reach level 5 of metamodeling quality. This topmost level means that all aspects of the language have been modeled, including its semantics. In this demo, semantics of ER was defined by using the translational approach, that is by translating ER concepts to Zope and AToM3 concepts that have precise semantics. Figure 3 shows these translations as Model→Code and Generate DSME activities. This illustrates that a language can have multiple semantics.

The demo also showed that modular transformations can be reused to form new transformation chains. Case in point is how ZCase was integrated in ER to Zope transformation by means of a relatively simple bridge. This is a counterexample to a claim that MDE solutions are inflexible.

Do you have experience with MDE applications within your organization? Can you share these experiences as well?

References

[1] Yu-Liang Chi. 2009. Ontology-based curriculum content sequencing system with semantic rules. Expert Syst. Appl. 36, 4 (May 2009), 7838-7847. 

While effort reduction and quality increase are both commonly recognized benefits of MDE, the former particularly has become its trademark, thanks to numerous generative uses in model-driven software development (MDSD). Examples include generation of code and configurations from models written in UML, DSLs and XML.

The_effort_in_mde_approach_with_partial_manual_coding

Figure: The effort in MDE approach with partial manual coding (adapted from [1])

The generative MDE automates well defined routine activities. An effective metric of depicting economical benefit thereof is effort. The above figure illustrates effort reduction due to automation and reuse in an MDE approach with partial manual coding. Of cause, the generative MDE improves quality as well: error reduction, enforced architecture conformance, and up-to-date documentation are common factors that have positive effect on software quality. But usually these are considered as icing on the cake that is effort reduction. In my experience this perspective on the economical value of MDE is common among both customers and MDE professionals. The perspective can be summarized as “the same with less”.

More with the same

Recently a client tasked me together with its domain experts to assess benefits of applying MDE to a difficult process within the organization. Having analyzed the before and after situations, we came up with estimated economical benefit expressed in effort savings. The estimate was hard to quantify, but “should” have been OK. Although I wanted to share this optimism, I felt that in practice the effort saving would be negligible if not even negative. This paradox was due to the fact that the largest activity in the problem domain was inherently creative and exploratory. 

In the figure, the output of a single exploration in this activity is shown as intermediate result, corresponding to line ad. As the figure suggests, code generation directly from the output is not possible (this happens further downstream in development). You may have noticed that the modelling curve rises more steeply towards point a. This rise occurs because modelling requires increased level of domain understanding and more information is needed by semantically rich operations, such as simulation, verification, code generation (eventually), etc. On the other hand, the figure shows effort reduction indicated by distance cd, which is the result of providing end users with proper abstractions, faster access to right knowledge, separation of concerns, DRY modelling, maintained consistency and integrity.

While working efficiency per exploration is likely to  increase (compare ab and cd), the leading concern is quality of the output. Here benefits are early detection of design errors, deep exploration of design choices, better communication and documentation, maximized reuse of domain-specific platforms in further development. Moreover, the domain experts noted that any saved effort would be re-invested in more alternative explorations in search of a more optimal output. This increased number of explorations, is likely to balance out any savings due to higher efficiency.

With these insights, economical benefits were expressed with quality metrics and linked to different business goals than initially thought.

Conclusion

The described MDE assessment targets a highly creative engineering activity that explores alternative choices. In extreme case, the main benefit is not effort reduction, but increased product and process quality. The icing on the cake is that processable models can open opportunities for generative uses as well.

In my experience, such and certainly less extreme quality-driven cases are not exotic. In recent years, quite a few MDE projects I’ve participated in, had benefits strongly linked to quality improvement. What are your MDE experiences with creative activities? What were the economical benefits and how were they conveyed?

References

[1] Thomas Stahl, Markus Voelter, Krzysztof Czarnecki. “Model-Driven Software Development: Technology, Engineering, Management”. Wiley; 1 edition (May 19, 2006)

L_from_pencils

Nowadays DSLs seem to be everywhere. If 5 years ago DSL was an exotic word in the UML dominated model driven world, today it has established a strong following. A recent research on how MDE is used in industry [1], indicated that nearly 40% of respondents use in-house DSLs (alongside of other languages). The in-house qualifier is important, as these DSLs are very likely to be developed with metamodels. In such cases, a quality benchmark may help language development. Yet, it is not easy to find such a benchmark, let alone one that is widely accepted.

 

Five levels of Metamodelling

One quality benchmark that I found useful is described by Tony Clarks et. al in [2]. The authors define 5 levels of quality. These briefly are:

  1. The lowest level: a simple abstract syntax is defined, but not implemented yet in a tool. The static and dynamic semantics of the language is informal and incomplete. There is no specific tool support: an existing language is repurposed, compliance with the DSL is manually maintained and models are mostly interpreted by users.
  2. At this level, the abstract syntax and static semantics have been largely defined, implemented in a tool and validated. The dynamic semantics is still informally defined.
  3. The abstract syntax is completely implemented and tested. Concrete syntax has been defined for the language, but not implemented yet. Optimization of the language architecture has started.
  4. The concrete syntax of the language has been implemented and tested. Users create models either visually and textually. The language architecture has been optimized for reuse and extensibility. Tool support for dynamic semantics begins to appear.
  5. The topmost level: all aspects of the language have been modeled, including its semantics. Models written in the language can be processed by the tool. Examples thereof include code generation, execution, simulation, verification. The language architecture is well optimized for reuse.

While the original intention of the benchmark was to assess metamodels, I found it also useful for estimating metamodelling capabilities of MD tools. If a tool is not capable of supporting development needs for a certain level, then that level will be the quality ceiling for all metamodels defined with the tool. In my experience, DSLs in traditional (fixed method) CASE Tools do not achieve level greater than 1. Metamodels in UML tools often do not reach level 4 (and often lack static semantics and concrete syntax). Language workbenches can typically produce level 5 metamodels.

Conclusion

The referenced benchmark provides a first order approximation of quality of metamodels. Furthermore, these 5 levels provide those looking for MDx technology, with a simple framework at least to question the marketing information by tool vendors. In my opinion this benchmark may be a useful ingredient in an answer to a more general question of comparing MD technologies.

What are your experiences with measuring quality of metamodels or comparing metamodelling capabilities of MDx tools? Which aspects are you interested in and how do you measure them?

References

[1] John Hutchinson, Mark Rouncefield, Jon Whittle, and Steinar Kristoffersen. Empirical Assessment of MDE in Industry. ICSE 2011.

[2] Tony Clark, Andy Evans, Paul Sammut, and James Willans. Applied Metamodelling: A foundation for Language Driven Development. Version 0.1. Xactium Ltd., 2004.

Image by Aqua