ATL survey

Quality of ATL model transformations

Description

The goal of our research is to make the quality of ATL model transformations measurable. To investigate the influences on the quality of ATL model transformations, we have to know what their perceived quality is. Therefore we would like you to participate in our experiment. We would like you, as an ATL expert, to evaluate the quality of (a number of) ATL model transformation(s) manually. The strength of the results of this experiment will benefit from the amount of model transformations that are evaluated. Filling out this survey will take approximately 10 to 15 minutes of your time.

Instructions

1.	Download one of the seven ATL model transformations and its description provided below. (Just pick one randomly.)
2.	Evaluate the transformation.
3.	Fill out the questionnaire provided below.

Transformations

Click to download

Survey

Background questions

1.	Please enter your name.
2.	Please enter your e-mail adress.
	We might use this to contact you with regard to your answers on the open questions.
3.	How would you rate your knowledge of ATL?
		Very Low		Low		Medium		High		Very High

4.	What is the number of the transformation you are evaluating?
	1	2	3	4	5	6	7

5.	Please enter the name of the transformation you are evaluating.
6.	How familiar were you with the model transformation you are analyzing?
		I have never heard about it		I have heard about it		I have seen it before		I have looked at it in detail		I have designed it

Evaluation questions

7.	What is the amount of effort required for the comprehension of the transformation?
	very little			average			very much
	1	2	3	4	5	6	7
8.	To what degree is the programming style in the transformation uniform?
	very low			average			very high
	1	2	3	4	5	6	7
9.	Could the transformation be written using fewer elements (but adhering to the same specification as the given transformation)?
	no, not at all			neutral			yes, definitely
	1	2	3	4	5	6	7
10.	To what extent do you think does the transformation provide the functionality it is supposed to provide?
	very low			average			very high
	1	2	3	4	5	6	7
11.	To what degree is the transformation (or parts of it) reusable to create other transformation?
	very low			average			very high
	1	2	3	4	5	6	7
12.	What is the expected difficulty of performing an average modification to the transformation?
	very low			average			very high
	1	2	3	4	5	6	7
13.	What is your expectation about the effort needed by a software engineer to understand the transformation well enough to be able to find errors in it?
	very low			average			very high
	1	2	3	4	5	6	7
14.	How would you rate the conciseness of the transformation?
	very low			average			very high
	1	2	3	4	5	6	7
15.	To what degree does the transformation contain conflicting elements?
	Think of things like conflicting variable names, etc.
	very low			average			very high
	1	2	3	4	5	6	7
16.	How useful is the transformation as a starting point for creating another (functionally related) transformation?
	not useful at all			neutral			very useful
	1	2	3	4	5	6	7
17.	How would you rate the modifiability of the transformation?
	very low			medium			very high
	1	2	3	4	5	6	7
18.	Can you give an indication of how well the transformation adheres to its specification?
	The question here is not to give this indication this, but if you could indicate it if required.
	very low			medium			very high
	1	2	3	4	5	6	7
19.	How much effort do you need to understand the purpose of an average rule or helper?
	Do not just try one.
	hardly any effort			medium effort			huge effort
	1	2	3	4	5	6	7
20.	How much effort is needed to adapt the transformation to support a change in one of the metamodels?
	hardly any effort			medium effort			huge effort
	1	2	3	4	5	6	7
21.	How would you rate the completeness of the transformation?
	very low			average			very high
	1	2	3	4	5	6	7
22.	How many elements of the source metamodel can be transformed by the transformation?
	none			average			all
	1	2	3	4	5	6	7
23.	To what degree is the transformation free of redundant or spare elements?
	Think of elements as variables, rules, etc.
	very low			average			very high
	1	2	3	4	5	6	7
24.	How much effort is needed to adapt the transformation in order to meet changing requirements?
	very little			average			very much
	1	2	3	4	5	6	7
25.	How would you rate the reusability of the transformation?
	very low			average			very high
	1	2	3	4	5	6	7
26.	How would you rate the consistency of the transformation?
	very low			average			very high
	1	2	3	4	5	6	7
27.	How would you rate the understandability of the transformation?
	very low			average			very high
	1	2	3	4	5	6	7

Open questions

In case you evaluate more than one case, these questions do not have to be filled in more than once.

28.	What affects understandibility of a model transformation in your opinion (either positively or negatively)?
29.	What affects modifiability of a model transformation in your opinion (either positively or negatively)?
30.	What affects completeness of a model transformation in your opinion (either positively or negatively)?
31.	What affects consistency of a model transformation in your opinion (either positively or negatively)?
32.	What affects conciseness of a model transformation in your opinion (either positively or negatively)?
33.	What affects reusability of a model transformation in your opinion (either positively or negatively)?
34.	If you have any other remarks, please put them in the textbox below

Template from freewebtemplates.com