Summary and Systemic Issues Report

Adult Multicultural Education Services

Introduction
Adult Multicultural Education Services (AMES) is the largest provider of English language and settlement
services in Victoria, in addition to being a major supplier of specialist employment and training services in
Melbourne.
AMES educates and services over 40,000 people every year, providing an extensive range of adult education,
training and employment programs to a diverse client community
In 1951, AMES began providing English to new settlers in Australia. More than half a million students have
attended AMES programs and services since its post-war inception. Prior to becoming an autonomous Adult
Education Institution, incorporated under the Adult Community and Further Education Act, AMES was a Service
Agency of the Victorian Government's Department of Education, Employment and Training.

Audit Methodology
AQTF audit
The audit process included three distinct phrases:
1. Desk audit
2. Site visit
3. Feedback from the VRQA.

1. Desk audit
The scope of the desk audit for re-registration included a review of:
 Institute annual report.
 Details of the Institute’s Quality System with particular emphasis on, but not limited to:
o A summary of how the Institute uses systematic and continuous improvement approaches in the
management of its operations (Ref Standard 3.1 AQTF 2007 Essential Standards for Registration)
o Decision-making systems that assure quality of educational delivery/assessment and
implementation across multiple campuses and delivery departments
o Details of processes that assure quality of teaching and learning provision, including teacher
selection procedures and teaching staff professional development
o The Institute’s scope of registration additions being supported by processes that comply with the
AQTF 2007 Essential Standards for Registration.
 A list of current delivery partnership arrangements, detailing the business name of the partner
organisation, qualifications delivered through the partnership, type of partnership (type of agreement,
delivery model – e.g. auspice/mixed mode etc, number of enrolments, fee for service, govt. profile etc).
Partnership list to also include VET in School arrangements.
 A summary of off-shore operations where VET/AQF qualifications are delivered. Detail to include
qualification, type of agreement, delivery model, e.g. full auspice/mixed mode etc, and number of
enrolments.
 Evidence of where the Institute is working towards adopting the AQTF 2007 Quality Indicators and
uptake of the SMART system for management of surveys, analysis and reporting. Ref:
http://www.acer.edu.au/aqtf/

/conversion/tmp/scratch/394301262.doc Page 1 of 9
Summary and Systemic Issues Report
Adult Multicultural Education Services

This submission was reviewed, and informed the audit sample for the second stage of the audit. The scope of
the audit was informed by the information provided in Stage one and analysis of the level of risk by the VRQA.

2. Audit visit
The second stage of the audit consisted of a visit to the Institute by two VRQA appointed auditors (Andrea
Bateman and Chloe Dyson). Audit visit was conducted over 2 days (24 & 25 September 2009), with a ½ day
devoted to each qualification.
The four qualifications audited were:
 TAA40104 Certificate IV in Training and Assessment
 BSB40401 Certificate IV in Business (Small Business Management)
 91423NSW Certificate III in Spoken and Written English
 SIT30707 Certificate III in Hospitality.
These four qualifications covered the following areas of provision:
 Sub-contractor (auspice) provision
 Traineeship provision.

3. Third stage
The third stage consists of a confidential audit report to the Institute CEO which includes an overview of the initial
submission and any systemic issues identified at audit. This report was accompanied by four individual audit
reports pertaining to the four qualifications.

Audit Findings: AQTF Review and Systemic Issues

Governance and strategic planning
AMES is an autonomous Adult Education Institution, accountable to the Victorian Minister for Skills and
Workforce Participation. It is governed by a Board established under the Education and Training Reform Act
2006. The AMES Board comprises seven members appointed by the Minister, three members co-opted by the
Board, one member elected by staff, and the AMES CEO.
Key documents related to strategic planning include:
 AMES Strategic Plan 2008–2010
 AMES Operational Plan 2009 (developed from the Strategic Plan) – annual
 Divisional Operational Plans – annual.
Strategic Plan and Annual Reports are accessible on the AMES website: http://ames.net.au/index.php?
sectionID=6547&pageID=6551

Management systems and continuous improvement

(AQTF Standards 1.1, 2.1, 3.1)
The AMES’ Quality Management System is certified to ISO 9001:2008 and maintained online with policies and
procedures, including forms and guidelines, across the scope of its operations (not just training and assessment
services).

/conversion/tmp/scratch/394301262.doc Page 2 of 9
Summary and Systemic Issues Report
Adult Multicultural Education Services

The QMS includes several levels:

 Corporate policies
 Corporate statements
 Procedures, guidelines and forms.

The QMS is sorted into key folders based on main operational areas:
 Administration
 Employment
 Finance
 Governance
 Human resources
 Occupational health and safety
 Program delivery
 Settlement services.

Key continuous improvement approaches include:

 Internal quality audits
 Quality Audit Plan 2008 is derived from AMES risk profile (Risk Management Plan 2008). This plan is
updated each year to meet emerging issues and risks. Quality Audit Plan 2008 includes mandatory
audits (AQTF, ISO and traineeship delivery). Staff members are trained as internal auditors for ISO
requirements, but AQTF audits are undertaken by only two quality auditors. Auditors utilise an audit tool,
which includes examples for evidence and opportunity to document audit evidence and findings.
 Management review of QMS
 AMES reviews the QMS at regular meetings as part of their continuous improvement. A brief schedule
was provided in the Submission Document.
 Suggestions for improvement.
Opportunities for improvement can be raised by staff via meetings and by raising continuous
improvement requests.
Clients’ opportunities include:
o Student Representative Group meetings, which the audit team considers a particular strength of the
organisation
o Client Satisfaction Surveys.
 Client and industry engagement
Industry involvement includes Student Representative Group meetings and local industry networks.
Note that training programs are intrinsically linked to other operations within AMES, e.g. settlement
services.

/conversion/tmp/scratch/394301262.doc Page 3 of 9
Summary and Systemic Issues Report
Adult Multicultural Education Services

Scope Management
(AQTF Standard 3.1)
AMES had in place the following documents in regards to scope management:
 PRG-30 Training package/Accredited Course Maintenance – Statement
 Curriculum Maintenance and RTO registration, PRG-30/1
 A guideline for updating AMES scope, PRG- 30 Guideline 1: Updating AMES Scope of Registration.

The procedure (PRG-30/1) references the Guideline in terms of adding to scope. The Guideline requires prior to
addition to scope the relevant business unit to confirm requirements:
 Staff are qualified and can demonstrate experience to level of competencies delivering
 Staff have Certificate IV in Assessment and Workplace Training (old qualification and it is suggested that
this title be updated)
 Staff have relevant industry experience
 Verified qualifications have been sighted
 Learning and assessment strategies are developed.

There is no requirement for the business unit to have in place training and assessment materials, or any
equipment and facilities that may be required.

Suggestion:
It is suggested that AMES consider more rigorous requirements prior to requesting additions to scope to ensure
that capacity to deliver is demonstrated prior to request for extension to scope.
This has been fully addressed. Specifically changed was the documentation format of the Teaching and Learning
Strategies and professional development conducted, to update staff knowledge.

Auspice arrangements
(AQTF Standard 3.2)
AMES provides a significant level of training via partners and sub-contractors. Partners are generally considered
co-signatories to contracts and sub-contractors may deliver under their own scope of registration or AMES scope
of registration. The following is a summary of agreements:
 Adult Migrant English Program (AMEP program): There are 4 partners and 40 sub-contract
arrangements with the qualification delivered being 91423NSW Certificate III in Spoken and Written
English
 Language Literacy Numeracy Program (LLNP): There is one partner who delivers under its own scope
of registration
 Traineeships: Auspice arrangements are in place for Certificate III in Hospitality and Certificate III in
Business.
 VCAL: There are two secondary schools conducting 91423NSW Certificate in Spoken and Written
English.

/conversion/tmp/scratch/394301262.doc Page 4 of 9
Summary and Systemic Issues Report
Adult Multicultural Education Services

Audited at AMES were three arrangements that were sub-contracting arrangements utilising AMES’ scope of
registration:
 91423NSW Certificate III in Spoken and Written English – Diversitat in Geelong, and Carlton
Neighbourhood House
 SIT30707 Certificate III in Hospitality – Mint Training.

There are no specific procedure, guidelines or forms within the QMS that address establishing, maintaining,
renewing and contract managing agreements; however,
 ADM-GL04 Checklist of Standard Clauses in AMES Agreements (revised 2008) states that the
agreement ‘must specify where responsibility for AQTF 2007 lies’. There is no guidance provided to
teams regarding how this is to be negotiated and documented.
 Procedure in PRG-25/1 identifies responsibilities for managing the contract, but is used for managing
the administration of traineeships.

Provided at audit were AMEP Consortium Partnership Principles, related to contract partners and sub-contractor
arrangements. However, these principles are more about selection principles than guidance in regards to
contract management and quality assurance arrangements.
Although all three arrangements reviewed had current contracts in place, there were issues in the level of detail
and requirements, especially:
 Quality assurance of training and assessment (AQTF elements 1.2, 1.3, 1.5)
 Ownership and copyright requirements in relation to the assessment materials (AQTF element 1.3)
 Collection of data and continuous improvement (AQTF elements 1.1, 2.1, 3.1)
 Management of complaints (AQTF element 2.6)
 Requirements to identify learners’ support needs and address these needs (AQTF element 2.4)
 How the qualifications of trainers will be checked and how evidence of vocational currency will be
maintained (AQTF elements 1.3 and 1.4)
 Provision of information to students about the AMES procedures and Trainee rights and responsibilities
that apply within the arrangement (AQTF 2.2, 2.5, 2.6)
 Arrangements for monitoring and reviewing the agreement (AQTF elements 3.1, 3.2).

Although in the VET program, AMES staff stated that Mr Stan Angelidis, Manager at AMES, has regular meetings
with Mint Training there were no minutes of these meetings provided at audit. Please note that Mr Angelidis was
unavailable for audit, due to leave arrangements.
For the CSWE agreements, although there was an AQTF matrix of roles and responsibilities, the few quality
assurance arrangements documented (e.g. provision of staff resumes on an annual basis) were not
implemented. However, for the CSWE programs in place were Field Officers who provide significant support and
quality assurance strategies; and there was significant reporting by parties to meet funding body requirements.
Across the three agreements, there was no evidence provided that the terms of the agreements were monitored
and reviewed.
Recommendation:

/conversion/tmp/scratch/394301262.doc Page 5 of 9
Summary and Systemic Issues Report
Adult Multicultural Education Services

AMES is required to develop and implement processes for the development, monitoring and review of auspiced
agreements so that these arrangements meet AQTF requirements.
This matter has been fully addressed. AMES no longer has auspiced delivery arrangements.

Staffing
(AQTF Standards 1.3, 1.4)
AMES has just implemented a performance management review of on-going staff and it has not been ‘fully rolled
out’ across the organisation.
However audit found:
 One CSWE trainer who has been working at the RTO for some time did not have the requisite
qualifications.
 Contract staff members are not addressed within the new performance management processes, and
there is no formal strategy in place at program manager level to ensure that contract staff will have on
file evidence of qualifications, experience and vocational currency.
 There was not a process in place to check that contracted trainers and trainers involved in auspiced
arrangements maintained their vocational currency.
 There was no formal process in place relating to trainers in auspice arrangements to confirm and retain
qualifications of sub-contractor staff.
Recommendations:
The RTO is required to develop and implement processes so that the currency of all trainers’ vocational
competence is systematically reviewed.
This matter has been fully addressed. Specifically implemented more stringent processes for recruiting new
teachers/trainers and for recording qualifications, experience & currency of teachers/trainers, curriculum vitaes
and skills matrices.
The RTO is required to develop and implement processes to ensure that confirm and retain qualifications of
trainers within auspice arrangements.
This matter has been fully addressed. AMES no longer has auspiced delivery arrangements.

The RTO is required to develop and implement processes to include contract staff within HR processes, to
ensure that qualifications and evidence of vocational competence are retained.
This matter has been fully addressed. Specifically the employment status for trainers has been changed to fixed
term or casual. Collection of qualifications and vocational competence are recorded in the same manner as for
teachers as stated above.

Strategies for training and assessment

(AQTF Standards 1.2)
Audit found that the RTO has just commenced developing strategies for training and assessment that are
contained in a cohesive document.
The AQTF does not mandate that a single strategy must be in place for each qualification – the strategy can be
drawn together from a range of documents. However, for three of the qualifications audited, the range of
documents provided did not provide a comprehensive or accurate picture of how the qualification was delivered
and assessed. The absence of an accurate strategy for training and assessment means that the delivery and

/conversion/tmp/scratch/394301262.doc Page 6 of 9
Summary and Systemic Issues Report
Adult Multicultural Education Services

assessment of a program cannot be effectively monitored and reviewed and that new staff members are not
provided with a true guide for the delivering and assessing the qualification.
Recommendation:
The RTO is required to ensure that accurate and comprehensive strategies for training and assessment are in
place for each of the VET qualifications audited. It is also strongly suggested that the RTO develop strategies for
training and assessment for all VET programs on its scope of registration.
This matter has been fully addressed. Specifically a Training and Assessment Strategy has been written for VET
programs being delivered and is accessible to all teachers/trainers on the AMES intranet. A standardised
template has been developed for documenting each of the Training and Assessment Strategies.

Assessment
(AQTF Standards 1.3 and 1.5)
Of the four qualifications audited, one had strong assessment processes and materials, two did not have
complete assessment tools in place, and one had assessment tools that required revision.
In addition, where the RTO was using an auspiced agreement or a contracted trainer to deliver a qualification,
the assessment resources were neither owned by AMES, nor quality-assured by the RTO, and the RTO did not
have a plan for the provision of training and assessment materials, should these arrangements cease. The
VRQA requires RTOs to have the capacity to deliver and assess all the qualifications on its scope of registration
and this includes having quality-assured training and assessment materials in place.
Audit found that VET trainers would benefit from professional development in the development of assessment
tools that meet Training Package or accredited course requirements.
Recommendation:
The RTO is required to develop and implement processes so that training and assessment materials that meet
the requirements of the Training Package/ accredited courses are in place for all relevant qualifications.
This matter has been fully addressed. Specifically teaching and learning materials have been developed for each
qualification by teachers and trainers working through network meetings. Teachers/trainers are trained in the
development and customisation of resource tools. All staff have upgraded their Cert IV TAE qualification which
has provided staff with current knowledge of validation requirements.

Assessment validation
(AQTF Standards 1.1 and 1.5)
Audit found that there was little evidence of assessment validation for the Vocational Education and Training
(VET) programs audited.
Evidence was provided of contract staff and staff involved in auspiced agreements carrying out assessment
validation external to the RTO, but the outcomes of this validation activity was not fed back into the RTO’s
processes.
PRG-05 Training moderation/validation provides a framework for validation, but it does not provide guidance on
planning for validation, for example, how many units should be validated and how to determine those that carry
the most risk. It does not provide an explanation of validation and it does not explain how validation outcomes
are subjected to a continuous improvement cycle. The ‘Moderation of Assessment tasks’ template provided at
audit assists assessors to ‘unpack’ a unit of competency, but it could be strengthened by leading them to:
 Critique the design of the tool: for example, is the assessment process clear? Does it have clear
instructions for candidates and assessors? Does it describe tasks that allow the assessor to make a

/conversion/tmp/scratch/394301262.doc Page 7 of 9
Summary and Systemic Issues Report
Adult Multicultural Education Services

judgement about competence? Are skills and knowledge requirements assessed in an integrated
fashion? Are there performance measures to guide assessors?
 Examine the application of the tool by comparing assessment judgements
 Record action taken as a result of assessment validation and monitor and review this action.
Recommendation:
The RTO is required to revise processes so that staff members are provided with guidance about their
responsibilities for validation of assessment and so that they meet these responsibilities. This includes contract
staff and staff involved in auspiced agreements.
This matter has been fully addressed. Specifically training and practice in the validation process is conducted at
VET/curriculum network meetings and trainer induction sessions. Auspiced training arrangements no longer
exist.

Records
(AQTF Standard 3.3)
For the CSWE student data is entered onto a DIAC database, ARMS, as part of the RTO’s contractual
requirements. However, this information is not entered into the RTO’s database, SHAPE. The ARMS database
does not produce AQF compliant awards and the RTO cannot use the ARMS data to produce reports relating to
the Quality Indicators.
The VRQA requires RTOs to store student records electronically for thirty years. The data that must be stored is:
 Family name and first name of student
 Residential postcode
 Date of birth
 Enrolment/commencement date
 Code and title of course
 Codes and titles of modules/units of competency
 Modules/units of competency assessment results
 Credentials and date issued.
Recommendation:
The RTO is required to develop and implement processes so that student records for all VET qualifications and
accredited courses are maintained according to VRQA requirements.
This matter has been fully addressed. Specifically the recording of client data procedure (TDM-19) available on
the Quality Management System, has been updated to address issues above and implemented. AMES no
longer uses SHAPE, now stores VET records on STRATA which is VRQA/NVR compliant.

Qualifications and Statements of Attainment

(AQTF Condition 6)
Audit found that a Statement of Results is not provided with a Certificate unless requested; therefore, the
qualification is non compliant as the AQF Implementation Handbook requires a listing of units attained to be
provided with each qualification.

/conversion/tmp/scratch/394301262.doc Page 8 of 9
Summary and Systemic Issues Report
Adult Multicultural Education Services

Audit also found that the Employability Skills statement on qualifications was included where it was not relevant
and that the AQF statement explaining the purpose of a Statement of Attainment is not included on Statements of
Attainment.
There is a procedure in place, Program Delivery Procedure PEG-07/2. Under the section about checking the
certificate it does not include guidance regarding revised requirements for issuing qualifications, for example,
Employability Skills and Statement of Attainment explanatory statement from the AQF Implementation Handbook.
The ARMS clients’ database does not print out compliant awards for CSWE. Students are provided with an
electronically generated Statement of Achievement; this statement does not hold the full title of modules attained
and their codes nor the AQF explanatory statement.
Recommendation:
Recommendations were made at qualification level.
This matter has been fully addressed. Specifically the procedure for the Production and Issuing of Qualifications
has been updated to address identified issues (TDM-07). It outlines the process to ensure compliant awards are
provided to clients. This is a centrally controlled process.

Good practice
Audit found that there were a number of good practices in place. These included the processes for moderating
assessment judgements in the CSWE; the panel assessment for the NEIS program; the use of Field Officers for
the CSWE program; quality assurance processes for the CSWE undertaken by the Field Officers; the use of the
extranet to provide external parties with forms; the operation of the Student Representative Group and the
collaborative and consultative processes described in the RTO’s procedures.
While there were issues related to the agreement in place for the Certificate III in Hospitality, this agreement
provided AMES students with excellent opportunities to gain industry experience and employment outcomes.
In addition, audit revealed strong student selection and support processes, especially for AMES managed
programs.

Conclusion
AMES has developed a strong quality management system for the continuous improvement of quality training
and assessment. In some instances, further review and revision of procedures is required; in others, greater
rigour in implementation and in monitoring is required.
The audit team reminds AMES that the recommendations embedded in this Summary Report are required to be
actioned, along with those recommendations noted in the four qualification reports.

Auditors:
Andrea Bateman
Chloe Dyson

Attachments:
AQTF Qualification Reports (4)

/conversion/tmp/scratch/394301262.doc Page 9 of 9