Organizacija, letnik 39 Predlogi za prakso številka 4, april 2006 Training Needs Analysis Tool Horovčak Pavel1, Lavrin Anton2, Orosz Marian3 ^Department of Applied Informatics and Process Control, Technical University of Kosice, Slovak Republic; Pavel.Horovcak@tuke.sk, Anton.Lavrin@tuke.sk 3Office of European Projects, Bozeny Nemcovej 3, 040 01 Kosice, Slovak Republic; Marian.Orosz@tuke.sk The contribution is focused on four steps of training needs analysis - general template creation, construct file creation, electronic questionnaire creation and evaluation of questionnaire results. It explains and specifies terms general template and construct file and their assignment in questionnaire design. Further it describes development, design and implementation of software support named "Query" for interactive creation of electronic questionnaires and their evaluation within the scope of some Leonardo da Vinci projects solution (in three phases). The software tool is based on utilization of PHP scripts, cascading style sheets and database system MySQL. Key words: template, questionnaire, questions types, evaluation, internet, PHP script Orodje za analizo potreb po učenju Prispevek je osredotočen na štiri stopnje analize potreb učenja: izgradnja splošne predloge, izdelava datoteke, izdelava elektronskega vprašalnika in ovrednotenje rezultatov vprašalnika. V prispevku je razložena in specificirana splošna predloga ter datoteka in njun namen v načrtu vprašalnika. Nadalje je opisan razvoj, načrt in vgradnja podporne programske opreme "Query" za interaktivno izdelavo elektronskih vprašalnikov in njihovo ovrednotenje z ozirom na nekaj Leonardo da Vinci projektnih rešitev (v treh fazah). Orodje programske opreme temelji na uporabi PHP skriptov, stopničastih zaslonskih listov in podatkovnega sistema MySQL. Ključne besede: predloga, vprašalnik, vrste vprašanj, ovrednotenje, splet, PHP skript 1 Introduction Training needs analysis, if done properly, provides the basis on which all other training activities can be considered. It is a process, which requires careful thought and analysis, which needs to be carried out with sensitivity. An individual's learning is important to him or her, and the success of the organization. The purpose of training needs analysis is to identify the skills gaps requirements and training needs of an individual or group of clients. The results of a needs analysis determine the scope and depth of required education. The main instrument of needs analysis is a questionnaire. Questionnaire design and evaluation has many theoretical and practical aspects. Many of them were discussed in frame of the first international meeting devoted to the methods used for questionnaire development, evaluation, and testing (QDET, 2002). The benefits of iterative questionnaire-evaluation research documents Esposito (2002). Independent and dependent interviewing compares and valuates Hoogendoom (2002), which for interviewing implementation has used the Blaise system - (Weer- man, 2001; also VIROS, 1999) in combination with Cent-ERdata's C2B software. Decisionarium (www.decisionarium.hut.fi) is the public site for interactive multicriteria decision support with tools for individual decision making as well as for group collaboration and negotiation. It is based on utilization of Java technology and designed by Hamalainen (2005). Our solution of training need analysis consists of following steps - general template creation, construct file creation (based on general template), electronic questionnaire creation (based on construct file, by means of interactive development tool "Query") and valuation of questionnaire results. 2 Templates as groundwork for questionnaire creation The general template was developed to determine progress development and training needs. For generalization 278 Organizacija, letnik 39 Predlogi za prakso številka 4, april 2006 of progress of work with needs analysis was the general template developed. The general template has the form of regular text document. The template consists of five sections and it is illustrated in [11]. The first section deals with customer identification, the second one is identification of business area(s) for organizational education and training. The third section focuses on education and training which could be offered by the partnership, the forth section aims to determine ICT skills. Section five concerns finance, contract conditions and contacts. The flexibility of the template created in this form is the ability to analyze a broad range of business customers. The Template can be used in several ways. The first way is to use the whole template with all five sections. The second way is to divide the template in two parts, which are processed separately. The first part consists of sections one and two. The second part of the template is created on the basis of obtained results and recognition of customer's needs as identified by sections one and two. This section aids the training organization to determine areas of training needs and enable the creation of customized courses. General training to be offered includes processes of change management, reengineering, audit, process modeling, business processes reengineering. The training is intended for distance form of education based on web based learning. 3 Construct file An intermediate file based on the general template has been created to facilitate questionnaire development. This file is entitled "the construct file". The form is similar to that of the template file. The first column of template file (Section) remains, the second one (Items) is divided in three columns, named No (Number of question), Question (Text of question) and Type of question (one of the values Textbox, Select, Radio button, Checkbox, Text area, Note). The Question column is based on column Items from template file, columns No and Type of question are new. The last column Answers in construct file corresponds to column Contents in template file. Example of such file is illustrated in [1]. Thus the construct file is then used as groundwork in process of questionnaire creation. 4 Interactive development tool "Query" for web questionnaire design The basic idea was to create software that undertakes the routine programming activities, related with questionnaire creation. This software for support of interactive creation of electronic questionnaire (named "Query") was utilized for questionnaire creation - section design, texts and types of questions, particular answers so as all the repairs and supplements - direct to sponsor. Questionnaire creation in this way is fundamentally quicker and easier. Designed software hides for its user amount of specialized internet technological problems, practices and solutions from the area of database systems (it operates with MySQL), the area of html technology, interactive scripts php, cascading style sheets css so as their mutual communication. Thus it enables the user to concentrate on the content aspect of electronic questionnaire creation and evaluation. A whole view of electronic questionnaire and its individual elements is solved on the base of consistent utilization of cascading style sheets (css). Cascading style sheets is saved in the independent external file, which is incorporated to result form of questionnaire by means of LINK command. The css file consists of named sections group, which are applied to the likewise named elements in the resulting html code of questionnaire. The coupling of individual elements of questionnaire to appropriate parts of style file is specified by means of matched statement . By means of this technique single classes are created (also in cascading manner). Named sections of style file are for example BODY, Nazov (name), Sekcia (section), Sekcianadpis (section title), Otazka (question), Cislo (number), Znenie (text), Odpoved (answer), select, input and Popis (description).The-se sections are the basic building elements (or modules) of resulting electronic questionnaire. A consistent system approach to questionnaire view control enabled entire isolation of view control from logic building-up and own functionality of questionnaire. During the designing of final questionnaire it is possible to choose one from more premeditates prepared style files (css), which ensures appropriate view. Comparatively single preparation of style file enables to achieve a unique view of each final questionnaire (and at the same time with wide variability). To authenticate the user, it is necessary to provide a username and password. Once the users have entered the login and password they should select a new or a previously created questionnaire. After that it follows the selection of questionnaire's language. It is now possible to create new section, new question and answers to the question. Texts of section, question and answers are in the construct file, from which they can be copied. The note to question is displayed before question, but it must be created only after the question's creation. Once the questionnaire is created, it is necessary to define questionnaire settings (option Setting in menu Settings). It is possible to choose its viewing (suitable css file), type title of questionnaire, define the time interval of questionnaire's accessibility, www page displayed after questionnaire filling and some other parameters. Option Preview in menu Settings enables to see the completed questionnaire. Further it is possible to add, repair or delete any section, question, note or answer (options Add or Edit in menu Editing). Option Edit enables also deleting of appropriate object. 279 Predlogi za prakso številka 4, april 2006 Organizacija, letnik 39 5 Questions types Questionnaire implementation exploits several miscellaneous types of html input elements for answers on questions. The first type is multiple choice, where the respondent is required to choose one item (tag Select). The second type is a pair of radio buttons with values Yes / No. The third type is a text input (tag input), in whose user can write the answer in one line. The fourth type is a check box, which can be indicated by mouse click. Each entry type has defined a default value (generally the first one), which is used when the user doesn't choose an answer. The fifth type is a multi-line text-input, where the user can write the answer in more lines. It is possible to assign a note to each question's type, which further specifies the question. 6 Language alternations For the purposes of the IMVOCED project (as well as other projects) it is necessary to present the questionnaire in different languages. Various questionnaire language versions can be realized in two ways. The first one exploits a replacement of questions and answers together with language code into the database table. From this table a selection for appropriate language selection is realized. The second option is based on the creation of appropriate script files for particular languages, from which the user can select the appropriate language. On the base of actual language selection, the questions file in particular selected language will be assigned for processing. 7 Evaluation of questionnaire Questionnaire evaluation (option Questionnaire in menu Evaluation) is offered through choosing a selection Questionnaire or Answer count (option Test is dedicated for valuation of test forms). Answer count enables the display of the frequency of answers to individual questions. Questionnaire is intended for presentation of filling questionnaires. List of questionnaires can be ordered on a base of specific attribute (for example name and surname) selected as the first option and filtered by means of the second criterion. A selection of other questions and one of the answers to this question represent the second criterion. In our tool "QUERY" there is the first approach used. 8 Completion of questionnaire The questionnaire is created direct on an URL. By typing this URL into a browser the questionnaire is ready for use. The respondent can type or select the appropriate answer for each question. After answering all questions and clicking the Submit button the answers are submitted and stored in the database table. Figure: 1. Illustration of questionnaire in phase I Figure: 2. Illustration of questionnaire in phase II 280 Organizacija, letnik 39 Predlogi za prakso številka 4, april 2006 9 Phases of solution The solution passed in three phases. On the IMVOCED project home page of the Technical University of Kosice there are three phases specified with characters I, II and III. The first phase (I) represents a standard way of questionnaire solution. An html page was manually created along with a database table for individual questionnaire data saving and some PHP scripts, which ensures saving the questionnaire into the database table, browsing and evaluation of questionnaires. Questionnaire selection for browsing is realized on the basis of name and address of organization. At the same time it was needed to solve a task of bilingual (eventually multilingual) form of questionnaire. Chosen solution worked using special variables for all questions and answers ($variable = "question text?"). Values of variables for given language were saved in independent file (langfile). To start the questionnaire it was necessary to choose a language. Based on this selection the appropriate file (langfile) was chosen. Adding new versions in other languages generate a whole host of problems which had to be overcome for the final version of the template. These problems included changing questions, adding or deleting questions and updating of tables, template and construct files. The changes made were complicated, repetitive and labour intensive. This gave rise to the idea of a system for automated creation and corrections of questionnaire. This first phase solution is on web page [1] (using [2], [3]), evaluation of questionnaire is in menu Administrator (Figure 1). The second phase (II) was based on the interactive approach. Texts of questions, answers and also sections using appropriate forms are saved (or corrected) into database tables. From these tables they are consequently selected into electronic questionnaire or into answers evaluation by means of group of scripts. Main functions of these scripts are creation of html questionnaire file (based on data in database tables), creation of PHP file for questionnaire saving as well as creation of PHP file for table generation. Any changes to the questionnaire's structure are transferred to appropriate tables in the database. The program automatically corrects corresponding files and scripts, removing the need for programming (or reprogramming) a questionnaire. This system can also be used by persons with no technical knowledge or background in internet usage. Therefore the investigator can create, correct and evaluate an electronic questionnaire. During this second phase solution the questionnaire was modified to create a template for the identification of training needs in TQM (Zeleznik, O. & Hurnâ) (based on template WP4Template2 [6]). This modification can be found on web page [4], its evaluation is on the web page [4] (Figure 2 below). The third solution phase III built upon the second phase, in terms of the user simplicity, flexibility and functionality point of view. For this reason a new interactive 3 http://ccdec.tukc,sk/-orosz/query/index, php?SSIWlf-lvorba/oprava otazky.php ^ WBÎ- withe learning support - Needs Analysis - Microsoft Internet txplorer |_ j [5] X Ete tfew FavortM loots fcMp » Links editing Add SECTION Add QUESTION Add NOTE Ed» SECTION Ed* QUESTION Ed* NOTE settings Setting Preview evaluation Quesbomare Test Answer ccurtt TESTIH >WP4 Addtingjage >ENG Register: IMVOCED III FffrHrffil Quezon 1. {1) - NJftK ûf organization: 2. (2)- Are you? 3. (3) ■ What is the constitution cf your organisai? 4. (4) - The mar area of business activity of your company is 5- (5) - How many employees are in you organisai*»!? 6. (6) - Podton 7. (7) • Area of activities 8. (8) - Cwrpetencies (resporrablbes) 9. (9) - Does you organization work with ISO 9QOQ/2QOO standards7 10. (10) - Does your organization work ISO L4001 standards? 11. (ll)-OoesyoworganizationworkwthBS3800standards? 12. (12) * Country (13) - Miat strategy does your organisation use in the area of maintenance? 2. (14) - Do you use norms of quaky m the area of mantenance? 3. (15) - In case yew answered yes, do yw use methods end prhcples cf sc-«âed (Total Productive Maintenance), that is part of TQM (Total Quatty Managements th 4. (16) - What are the strengths of the maintenance in your organisation accordng 5. (17) - Wwt are the weaknesses of maintenance in your organisation occcnSnQ y 6- (20) - How many suppfcrs have organisation ri maintenance area? • Entemet Ne Erf* View Favcftes Tools HHp iff Address [ffi rytîs.^?m=IMyOŒ[^=V^4kiozyk=ENGfass=cs^ Q Go ünb " WBL With E-Learning Support - Needs Analysis Section I. CUSTOMER IDENTIFICATION 1. Name of organization:_ C 2. Are you? I Company 3. What is the constitution of your organisatio? [Small trader v 4.The main area of business activity of your company is [Production: v 5. How many employees ore in your organisation? O < 10 (Micro) O 10-19 (Small) O 20-49 (Small) O 50-99 (Middle) O 100-249 (Middle) 0 250 < (Large) Personal Information.s About Person Responsible for Answers 6. Position 1 nirKnlnr vl Don? Figure: 3 Interactive development tool "QUERY" Figure: 4 Output of "QUERY" as questionnaire WP4 281 Organizacija, letnik 39 Predlogi za prakso številka 4, april 2006 development tool "QUERY" was developed for web questionnaire design (see Figure 3). Tool "QUERY" supports the interactive creation of an electronic questionnaire which enables the user to create new questionnaires, edit, correct or add existing questionnaires (for example the change of question/answer text, change of question/answer order, question/answer adding or deleting and similarly for sections and also for questionnaire question's description). This system is accessible free of charge (Orosz, 2004). Using the tool "QUERY for interactive creation of electronic questionnaire some new questionnaires were created. The first one is a questionnaire for investigating of evaluation of education process results in the TQM area in frame of project IMVOCED. The questionnaire (named TEST03) was created in two language versions (English [8] and Slovak [9]). Evaluation of questionnaire TEST03 is accessible through option Questionnaire in menu Evaluation in development tool (Orosz, 2004). Questionnaire TEST03 is located on web page [7]. The second questionnaire is a general questionnaire for purposes of identification of needs analysis for WBL with E-learning support, which was named WP4 and is located on URL [10]. This questionnaire was created utilizing the general template [11] on the base of derived construction file Construct [12] (see Figure 4). The third questionnaire is an adapted questionnaire for purposes of identification of needs analysis for WBL with E-learning support in the TQM area. The questionnaire is named TQM and is located on URL [13]. This questionnaire was also created utilizing the general template [14] on the base of derived construction file Construct [15]. The fourth questionnaire deals with reconnaissance of interest in training needs in area of TQM (September 2001). It is named WP1&2, it was applied in 8 companies and it is located on URL [16]. The manner of evaluation of questionnaires is described in quick user's guide [17]. The fifth questionnaire deals with quality valuation of learning materials created in frame of ICOTEL project. Questionnaire will be applied in 10 companies and it is located on URL [18]. Other application of the questionnaire (Horovcâk, 2005) will be applied for needs analysis in the same project as well. 10 Conclusion Exploitation of electronic questionnaires for market reconnaissance is currently much augmented activity on the side of all producers and suppliers of miscellaneous products and services (Peter and Donnelly, 1992; Delina and Lavrin, 2003). Designed tool "QUERY" for interactive creation of electronic questionnaire enables electronic questionnaire realization without requirement on technical, technological and programming knowledge, what enables its exploitation in a wide range. The main preferences of designed system are the automation and interactivity of creation process of new electronic question- naire, a simplification of editing, corrections, adding or removing of questions or answers for existing questionnaire as well as multilingual support enabling creation of various language mutations of the same electronic questionnaire. Applications of questionnaires confirmed good performance of developed tool. In the future there will be necessary to make-up some supporting services such as Help, user guide, installation guide. Future development we expect in area of application's exploitation extension in other areas of investigation what will require new methods of questionnaires formation and valuation. Notice: The contribution was solved in frame of projects KEGA 3/3084/05, KEGA 1/3126/05 (B), KEGA 1/3124/05 (L) and ICOTEL (LdV) SK/02/B/F/PP/-142261. Literature Delina, R. & Lavrin, A. (2003). Approach to the performance measuring of e-commerce solutions in manufacturing firm. Proceedings of ICIL'2003, Vaasa Finland 2003, ICIL Southampton UK (www.iciil.org), ISBN 952-476-0101-X, pp. 130 - 139 Esposito, J.L. (2002). "Iterative, Multiple-Method Questionnaire Evaluation Research: A Case Study." The International Conference on Questionnaire Development, Evaluation and Testing Methods, 14-17 November 2002, Charleston, SC. Available from http://www.jpsm.umd.edu/qdet/final pdf papers/Esposito.pdf (Accessed 11.2.2006) Hamalainen, R.P. (2005). Decisionarium - Aiding Decisions, Negotiating and Collecting Opinions on the Web. Available from http://www.sal.hut.fi/Publications/pdffiles/mham03. pdf(Accessed 11.2.2006) Hoogendoorn,A. (2002). „Evaluation of a Questionnaire Design for Dependent Interviewing in a Web Survey." The International Conference on Questionnaire Development, Evaluation and Testing Methods, 14-17 November 2002, Charleston, SC. Available from http://www.jpsm.umd.edu/qdet/fi-nal pdf papers/round%20three/Hoogendoorn.pdf (Accessed 11.2.2006) Horovcák, P. (2005). Comparison of gender relation to information and communication technologies at the BERG Faculty (in slovak). Acta Montanistica Slovaca ?. 3,2005, volume 10, ISSN 1335 - 1788, pp. 331 - 336 Available from http://acta-mont.tuke.sk/pdf/2005/n3/10horovcak.pdf (Accessed 11.2.2006) Orosz, M. (2004). Interactive development tool "QUERY" for web questionnaire design. Available from: http:// 147.232.1.101/query/ (Accessed 15. November 2005) Peter, J.P. & Donnelly, J.H. (1992). Marketing management: Knowledge and skills.3rd. ed. Burr Ridge, Irwin, 943 pp., ISBN 0-256-09225-7 QDET (2002). International Conference on Questionnaire Development, Evaluation and Testing Methods, 14-17 November 2002, Charleston, SC. Available from http://www.jpsm. umd.edu/qdet/qdet-set.html (Accessed 11.2.2006) VIROS (1999). Virtual Institute for Research in Official Statistics (Eurostat) ETK'99 - Agenda. Available at http://euro-pa.eu.int/comm/eurostat/research/index.htm?http://euro-pa.eu.int/en/comm/eurostat/research/conferences/etk-99/agenda.htm&1 (Accessed 11.2.2006) 282 Organizacija, letnik 39 Predlogi za prakso številka 4, april 2006 Weerman, B. (2001). Internet interviewing using Blaise API. Available from http://www.blaiseusers.org/ibucpdfs/2001/ Weerman--IBUC Paper.pdf (Accessed 11.2.2006) Zelezník, O. & Hurná, S. (2001). Innovation management from TQM to reengineering (in slovak). In: Strojárstvo v hospodárstve a priemysle. roc. 5, c. 7+8 (2001), s. 38-39. Sources [2] Questionnaire Q1 Training Needs Analysis Template Available from: http://ccdec.tuke.sk/chorovcak/php db/imvoced/ (Accessed 15. November 2005) [3] Template file for Q1. Available from: http://www.tuke.sk/imvoced/Doc/questionnaire.doc (Accessed 15. November 2005) [4] Php template file for Q1. Available from: http://www.tuke.sk/imvoced/Doc/texty template.phpx (Accessed 15. November 2005) [5] Questionnaire Q2 IMVOCED Entry Questionnaire Available from: http://ccdec.tuke.sk/chorovcak/php db/imvo1 (Accessed 15. November 2005) [6] Evaluation of Questionnaire Q2. Available from: http://ccdec.tuke.sk/chorovcak/php db/imvo1/info.html (Accessed 15. November 2005) [7] Template file for Q2. Available from: http://www.tuke.sk/imvoced/Doc/WP4Template2.doc (Accessed 15. November 2005) [8] Questionnaire Q3 Overall Evaluation of Course (two language mutation) Available from: http://147.232.101/query/start/query1.php?m=IMV0CED &t=TEST03&jazyk=ENG (Accessed 15. November 2005) [9] Template file for Q3 Eng. Available from: http://www.tuke.sk/imvoced/Doc/WP4TemplateEntryENG. doc (Accessed 15. November 2005) [10] Template file for Q3 Svk. Available from: http://www.tuke.sk/imvoced/Doc/WP4TemplateEntrySVK. doc (Accessed 15. November 2005) [11] Questionnaire Q4 WBL with E-learning Support - Needs Analysis General Available from: http://147.232.101/query/start/query1.php?m=IMV0CED &t=WP4&jazyk=ENG (Accessed 15. November 2005) [12] Template file for Q4. Available from: http://www.tuke.sk/imvoced/Doc/WP4Template.doc (Accessed 15. November 2005) [13] Construct file for Q4. Available from: http://www.tuke.sk/imvoced/Doc/WP4Construct.doc (Accessed 15. November 2005) [14] Questionnaire Q5 WBL with E-learning Support - Needs Analysis General Available from: http://147.232.1.101/query/start/query1.php?m=IMV0CE D&t=TQM&jazyk=ENG (Accessed 15. November 2005) [15] Template file for Q5. Available from: http://www.tuke.sk/imvoced/Doc/WP4TemplateTQM.doc (Accessed 15. November 2005) [16] Construct file for Q5. Available from: http://www.tuke.sk/imvoced/Doc/WP4ConstructTQM.doc (Accessed 15. November 2005) [17] Questionnaire Q6 Questionnaire of improving vocational education in TQM Available from: http://147.232.1.101/query/start/query1.php?m=IMV0CE D&t=WP12&jazyk=ENG (Accessed 15. November 2005) [18] Quick guide (Evaluation) for Q6. Available from: http://www.tuke.sk/imvoced/Doc/Quick Users Guide.doc (Accessed 15. November 2005) [19] Questionnaire Q7 Questionnaire of Icotel SK/02/B/F/PP/-142261 (http://icotel.ktl.elf.stuba.sk/) Available from: http://147.232.1.101/auerv/start/winauerv1.php?m=ICQ-TEL&t=QUESTIONNAIRE1&jazyk=SVK (Accessed 15. November 2005) Pavel Horov~ák is Associate Professor of applied informatics at Technical University of Kosice, Slovak republic, Faculty of Mining, Ecology, Process Control and Geotechno-logy. He graduated from Faculty of Electrical Engineering, TUK, in Technical Cybernetics in 1972. Since 1972 he has worked as a researcher and lecturer at several faculties/institutions at the Technical University in Kosice (in the areas of: control engineering, graphical presentation of processes, information, monitoring and control systems development and management, and distance education). He has participated in several international projects (Leonardo da Vinci, Socrates, ICoTeL). He delivers lectures in Informatics, Algorithm Process and Programming, Programming in C, Pascal, Java, Real Time Systems, Internet Technologies. Dr. Horovcák published a number of papers in journals related to application of ICT. Anton Lavrin Associate Professor has been working as a researcher and lecturer at several faculties/institutions at the Technical University in Kosice (in the areas of: control engineering, information systems development and management, TQM and software quality, innovation management and distance education). He held several managerial positions like: Vice-rector for informatics, Director of the Institute of Life-long Education. Dr. Lavrin has participated in several international projects (Tempus Phare, Leonardo da Vinci, Socrates, 5FP and 6FP). Currently he is the delegate / member of the ISTC / on behalf of the Slovak Republic. Marian Orosz He graduated from Faculty of Mining, Ecology and Geotechnology, Technical University of Kosice, in field Process Informatization. Today is Member of staff on Technical University of Kosice at Office of European Projects - representation in advisory and consultant services guaranteed by department according to approved specification and participate in establishment of the international university undertaking. He participates on ERA Slovakia project as a web administrator of the Slovak national mobility portal. 283