Quality of ATL model transformations

Description

The goal of our research is to make the quality of ATL model transformations measurable. To investigate the influences on the quality of ATL model transformations, we have to know what their perceived quality is. Therefore we would like you to participate in our experiment. We would like you, as an ATL expert, to evaluate the quality of (a number of) ATL model transformation(s) manually. The strength of the results of this experiment will benefit from the amount of model transformations that are evaluated. Filling out this survey will take approximately 10 to 15 minutes of your time.

Instructions

1.         Download one of the seven ATL model transformations and its description provided below. (Just pick one randomly.)
2. Evaluate the transformation.
3. Fill out the questionnaire provided below.

Transformations

Click to download

Transformation 1 Transformation 2 Transformation 3 Transformation 4 Transformation 5 Transformation 6 Transformation 7

Survey

Background questions



1. Please enter your name.
 
 
2. Please enter your e-mail adress.
We might use this to contact you with regard to your answers on the open questions.
 
 
3. How would you rate your knowledge of ATL?
Very Low Low Medium High Very High
 
4. What is the number of the transformation you are evaluating?
 
1 2 3 4 5 6 7
5. Please enter the name of the transformation you are evaluating.
 
 
6. How familiar were you with the model transformation you are analyzing?
I have never heard about it I have heard about it I have seen it before I have looked at it in detail I have designed it


Evaluation questions



7. What is the amount of effort required for the comprehension of the transformation?
very little average very much
1 2 3 4 5 6 7
 
 
8. To what degree is the programming style in the transformation uniform?
very low average very high
1 2 3 4 5 6 7
 
 
9. Could the transformation be written using fewer elements (but adhering to the same specification as the given transformation)?
no, not at all neutral yes, definitely
1 2 3 4 5 6 7
 
 
10. To what extent do you think does the transformation provide the functionality it is supposed to provide?
very low average very high
1 2 3 4 5 6 7
 
 
11. To what degree is the transformation (or parts of it) reusable to create other transformation?
very low average very high
1 2 3 4 5 6 7
 
 
12. What is the expected difficulty of performing an average modification to the transformation?
very low average very high
1 2 3 4 5 6 7
 
 
13. What is your expectation about the effort needed by a software engineer to understand the transformation well enough to be able to find errors in it?
very low average very high
1 2 3 4 5 6 7
 
 
14. How would you rate the conciseness of the transformation?
very low average very high
1 2 3 4 5 6 7
 
 
15. To what degree does the transformation contain conflicting elements?
Think of things like conflicting variable names, etc.
very low average very high
1 2 3 4 5 6 7
 
 
16. How useful is the transformation as a starting point for creating another (functionally related) transformation?
not useful at all neutral very useful
1 2 3 4 5 6 7
 
 
17. How would you rate the modifiability of the transformation?
very low medium very high
1 2 3 4 5 6 7
 
 
18. Can you give an indication of how well the transformation adheres to its specification?
The question here is not to give this indication this, but if you could indicate it if required.
very low medium very high
1 2 3 4 5 6 7
 
 
19. How much effort do you need to understand the purpose of an average rule or helper?
Do not just try one.
hardly any effort medium effort huge effort
1 2 3 4 5 6 7
 
 
20. How much effort is needed to adapt the transformation to support a change in one of the metamodels?
hardly any effort medium effort huge effort
1 2 3 4 5 6 7
 
 
21. How would you rate the completeness of the transformation?
very low average very high
1 2 3 4 5 6 7
 
 
22. How many elements of the source metamodel can be transformed by the transformation?
none average all
1 2 3 4 5 6 7
 
 
23. To what degree is the transformation free of redundant or spare elements?
Think of elements as variables, rules, etc.
very low average very high
1 2 3 4 5 6 7
 
 
24. How much effort is needed to adapt the transformation in order to meet changing requirements?
very little average very much
1 2 3 4 5 6 7
 
 
25. How would you rate the reusability of the transformation?
very low average very high
1 2 3 4 5 6 7
 
 
26. How would you rate the consistency of the transformation?
very low average very high
1 2 3 4 5 6 7
 
 
27. How would you rate the understandability of the transformation?
very low average very high
1 2 3 4 5 6 7


Open questions

In case you evaluate more than one case, these questions do not have to be filled in more than once.



28. What affects understandibility of a model transformation in your opinion (either positively or negatively)?
 
 
 
29. What affects modifiability of a model transformation in your opinion (either positively or negatively)?
 
 
 
30. What affects completeness of a model transformation in your opinion (either positively or negatively)?
 
 
 
31. What affects consistency of a model transformation in your opinion (either positively or negatively)?
 
 
 
32. What affects conciseness of a model transformation in your opinion (either positively or negatively)?
 
 
 
33. What affects reusability of a model transformation in your opinion (either positively or negatively)?
 
 
 
34. If you have any other remarks, please put them in the textbox below