Report on the Evaluation of the Application Process — DocMus.pt 2026/2027
Results of the follow-up survey to candidates in the 1st edition of the Doctorate in Music — Artistic Practices in the Portuguese Context
Results of the follow-up survey to candidates in the 1st edition of the Doctorate in Music — Artistic Practices in the Portuguese Context
At the end of the application process for the 1st edition of DocMus.pt (2026/2027), the Coordination of DocMus.pt sent all candidates a survey evaluating the application process, with the aim of identifying what worked well and what should be corrected before the next edition. This report systematises the responses received and publicly and transparently acknowledges the aspects in which the process fell short of what is intended for the programme.
The survey was answered anonymously. In this first application, 17 responses were obtained, corresponding to a response rate of approximately 61% of the universe of 28 candidates admitted to Phase 1. This is a response rate considerably above what is usual for surveys of this kind, which lends the results a relevant degree of representativeness in relation to the universe of candidates — without prejudice to the fact that individual figures should be read with the caution warranted by a still limited absolute sample size.
The survey covered six dimensions of the application process: clarity of the call for applications and admission criteria; sufficiency of the information made available; ease of use of the application platform; the process for submitting the Preliminary Research Project (PPI) and letters of recommendation; the quality of support provided to candidates; and the transparency and timeliness of communication of results. Most questions used a 1-to-5 agreement scale (1 = strongly disagree; 5 = strongly agree); two multiple-choice questions asked candidates to identify the aspects that worked best and worst; three questions were open-ended.
This section presents, for each dimension assessed, the exact question put to candidates, the average obtained on a 1-to-5 agreement scale (1 = strongly disagree; 5 = strongly agree) and a brief reading note. The dimensions are ordered from the highest score to the lowest; the two marked in amber are those the Coordination of DocMus.pt identifies as priorities for correction, discussed in their own right in section 6.
"The speed and usefulness of the responses to questions raised"
The most highly rated dimension of the entire survey — a direct indicator of the effectiveness of the candidate support service (see section 5).
"Deadlines and communication about deadlines were adequate."
One of the dimensions with the highest agreement in the entire survey (see section 5).
"The availability and clarity of the contact information for support"
Confirms the robustness of the candidate support channel throughout the process.
"The call for applications and the admission criteria were clear."
Rated well overall; the specific exceptions identified by candidates are detailed in section 6.
"The technical support provided"
Consistent with the other candidate-support dimensions.
"Communication about decisions and results"
Rated well, albeit slightly below the other support dimensions.
"The application platform was easy to use."
Positive quantitative rating; the specific technical aspects identified as less successful are detailed in section 6.1.
"The candidate support service responded to questions in a timely manner."
Rated well, with no qualitative complaints associated.
"I would recommend the programme to other candidates, based on my experience of the application process."
Summary indicator of the overall experience: 13 out of 17 candidates agree or strongly agree.
"The information available about the programme was sufficient to prepare the application."
Positive overall; complements the discussion of specific information gaps regarding the Digital Artistic Portfolio (section 6.2).
"The information received from official sources was sufficient to prepare the application."
Confirms the previous assessment of the general sufficiency of the information provided.
"The submission form for the Preliminary Research Project (PPI) was clear regarding the requirements (format, length, required content)."
Positive rating, which contrasts with the lower perceived clarity regarding the Digital Artistic Portfolio (section 6.2).
"The evaluation criteria were applied transparently."
Discussed in detail in section 6.2.1, including a clarification of the role of the panel's diversity of judgement.
"The time available was sufficient to prepare an application of the desired quality."
The lowest-rated dimension outside the two flagged as priorities; related to the late timing of the process, discussed in section 6.5.
"The admission decision was communicated in a timely manner."
Second-lowest agreement in the survey. Discussed in section 6.4.
"The process for requesting and submitting letters of recommendation through the platform worked without problems."
The lowest-agreement dimension in the entire survey. Discussed in section 6.3.
Three further questions, with categorical responses and not included in the table above, complete the set of survey dimensions:
"How long did it take you to prepare your application?"
Balanced distribution across the proposed intervals, with no dominant pattern: 24–72h (2 responses); 72h to 1 week (3); 1 to 2 weeks (4); 2 to 4 weeks (4); 1 to 3 months (4).
"Compared with other application processes for doctoral programmes, this process was..." (1 = much more difficult; 5 = much easier)
Most responses (10 out of 17) fall at the midpoint of the scale — with no clear sense that the process was either easier or more difficult than at other institutions.
"Would you like to receive feedback on your application?"
Unanimous response ("Yes" in every response), with operational follow-through in section 8.
Candidates who expressed lower agreement with the clarity of the call for applications were asked to specify in what respect(s). Seven of the 17 responses identified at least one aspect, distributed as follows:
| Documents required for the application | 4/17 | |
| Expected format of the Digital Artistic Portfolio | 1/17 | |
| Competition calendar (phases, deadlines) | 1/17 | |
| Places available and conditions for opening the programme | 1/17 | |
| Evaluation/selection criteria | 1/17 |
The most frequently flagged item — "documents required for the application" — aggregates responses of a distinct nature: in some cases it refers directly to the lack of clarity regarding the Digital Artistic Portfolio, already addressed in detail in section 6.2; in other cases it was not accompanied by open-text comments allowing the specific document in question to be identified. The calendar and the places available are discussed in section 6.5. As for "evaluation/selection criteria", the response was not accompanied by open-text comments; the Coordination of DocMus.pt notes the mention, but refers its interpretation to the note on the transparency of the evaluation criteria (section 6.2.1).
Multiple-choice question — candidates could select more than one option.
| Quality of support and guidance | 11/17 | |
| Transparency of the evaluation criteria | 7/17 | |
| Communication and information provided | 7/17 | |
| Ease of use of the platform | 7/17 | |
| Speed of the process | 6/17 | |
| Clarity and rigour of the call for applications | 5/17 |
| Transparency of the evaluation criteria | 5/17 | |
| Ease of use of the platform | 5/17 | |
| Communication and information provided | 2/17 | |
| Clarity and rigour of the call for applications | 1/17 | |
| Time to prepare materials and their clarity | 1/17 | |
| Speed of the process | 1/17 |
Responses converge consistently on three strengths of the process, which the Coordination of DocMus.pt intends to preserve in future editions:
Cross-analysis of the quantitative and qualitative responses reveals five recurring problems, several of them raised spontaneously and independently by different candidates — which reinforces their relevance.
Ease of use of the platform is tied with the transparency of the evaluation criteria at the top of the aspects identified as least successful in the process (5 out of 17 mentions each). The quantitative rating for this dimension is nonetheless positive overall (4.35/5).
"Make sure the platform works in all its functions (e.g. it is not possible to insert 2 letters of recommendation, or two pdf files)"Survey response
The complaints received about the platform share a common cause: submission fields that admit more than one valid way of being filled in, leaving it to the candidate to decide which one satisfies the requirement. This is the case with the letter-of-recommendation field, detailed in section 6.3. It is also the case, identified directly by the Coordination of DocMus.pt outside the scope of this survey, with the Curriculum Vitae submission field: several candidates entered the link to their Ciência Vitae profile instead of uploading the document in PDF — an alternative that the platform allows. The Coordination of DocMus.pt considers that, in the next edition, this should become the required format for this field, replacing the PDF upload.
The underlying problem lies, therefore, in the absence of a closed form architecture, free of scope for interpretation, for each requirement of the application, and goes beyond the lack of multiple uploads in any one isolated field. The Coordination of DocMus.pt considers that it should implement this architecture, replacing field-by-field fixes, before the next edition.
Three candidates noted, independently and without the question suggesting it directly, difficulty in understanding what was intended by the Digital Artistic Portfolio (PAD), its expected format, or the mark it was given.
"I would ask that you clarify, in detail, the specific model for a Digital Artistic Portfolio. As with the model made available for the Research Proposal, there should be a single model, with format rules, so that there is no ambiguity in this area."Survey response
"[I]n the Digital Portfolio I found no guidance on what they wanted. They could have provided a model, or a list of criteria to be assessed. [...] I produced a Portfolio that was praised by my peers [...], yet I was surprised to receive a low mark. To this day, I do not understand the reasons that justify the mark I was given." [sic]Survey response
"Expected portfolio format: the selection of a limited number of works and the brief description suggest a short document, but the page limit suggests otherwise" [sic]Survey response
On reviewing the text of the call for applications, the Coordination of DocMus.pt finds that the requirements for the PAD were defined. As to format, the call provides for a "single document (PDF), of max. 50 pages" or, "alternatively", a "personal web platform, with a curated portfolio (max. 5 pages) accessible via a permanent URL". As to content, it specifies a "brief description of 3–5 main artistic works/projects from the last 5 years, with photographs, technical descriptions, dates", "links (URLs) to audio/video recordings, where available" and a "brief critical reflection (max. 5,000 characters with spaces) on the role of artistic practice in one's personal trajectory". As to the evaluation criteria, the call states that "the portfolio is assessed for consistency, thematic relevance and technical-aesthetic quality", detailing five specific criteria — consistency, relevance, technical quality, innovation/experimentation and documentation — and a qualitative grid by mark intervals (0–5, 6–10, 11–15, 16–20). The reading of the complaints received suggests that, in most cases, the source of the problem lay in an incomplete reading of the call and in an interpretation of the PAD as an extension of the curriculum vitae, despite the call defining it as a distinct document, of a curatorial and critical nature.
There remains, nonetheless, an objective tension in the signalling of the call for applications, which the responses themselves reveal. The quotation above shows one candidate reading the 50-page limit as an invitation to produce an extensive document, in contradiction with the "brief description" of 3–5 works requested. Another candidate read the same parameter in the opposite sense, considering it too restrictive to accommodate scores and texts:
"If you keep the pdf format, limiting the file to 50 pages, when scores and texts are required, makes it very restrictive when it comes to including these elements."Survey response
The latitude of format allowed for the PAD is deliberate: the document has to accommodate candidates from areas with materials of a very different nature — from performance to composition, by way of production and music technology —, allowing, for example, a composition candidate to include scores in PDF and a performance candidate to link to audio or video recordings. The 50-page ceiling is sized to accommodate these edge cases and constitutes an admissible maximum, with no implied recommendation as to the desirable length. Opposing readings of the same parameter indicate, however, that this logic, being implicit, needs to be made explicit in the text of the next call for applications.
This responsibility should, however, be shared with the design of the process. The PAD is, by the design of the competition, the only instrument available to the panel to directly assess the candidate's artistic value and quality — the Curriculum Vitae assesses professional trajectory, standing and recognition. This distinction in function between the two documents, although stated in the call, proved insufficiently evident in practice for several candidates, which the Coordination of DocMus.pt regards as a design weakness of the process to be corrected.
The measure to be implemented is the replacement of the free-form format with a closed form for the PAD, with structured fields that require the information to be presented in the format and with the curatorial-critical nature intended, reducing the risk of the document being read — by either the candidate or the panel — as a duplicate of the CV.
The transparency of the evaluation criteria was identified both as a strength (7 mentions) and as a weakness (5 mentions), and registers moderate agreement (4.06/5). The Coordination of DocMus.pt considers it necessary to clarify publicly a fundamental point: the evaluation parameters and the corresponding marking grid are set out in the call for applications and are common to all members of the panel; in a competition of an artistic nature, however, their application does not require mechanical uniformity of reading among assessors. It is appropriate and desirable that, within the shared parameters, different members of a deliberately broad panel — five full members — should weigh the different strengths and weaknesses of each application differently; it is this diversity of judgement that ensures a robust artistic assessment. Transparency of process therefore means that the general evaluation parameters are known in advance and applied with rigour and internal consistency to each application — which the Coordination of DocMus.pt considers to have been the case — and is entirely compatible with the diversity of individual judgement proper to an artistic panel.
This is the quantitative dimension with the lowest agreement in the entire survey (3.47/5, with 4 responses of disagreement, including one strong disagreement). Several candidates described, consistently with one another, the same point of confusion regarding the manner of submitting letters of recommendation — the same form-architecture problem discussed in section 6.1.
"[A] field appears for submitting only one letter of recommendation when up to two could be submitted. This makes this step of the submission confusing [...]"Survey response
"[I]t is not clear that these [letters of recommendation] have to be submitted by the person recommending."Survey response
The call for applications establishes that letters of recommendation are "optional, but recommended — maximum: 2" and that "submission will be made directly by the referee, electronically, through a dedicated form provided by the institution", adding that "the procedure will ensure the confidentiality and traceability of the documents submitted". The candidate therefore does not submit the letters — they merely indicate the referees, who submit them directly. This procedural design is intended to preserve the confidentiality of the letter from the candidate, allowing the referee to choose whether or not to share its content with the person being recommended. The Coordination of DocMus.pt confirms that the application platform nonetheless came to present candidates with a field for direct upload of a letter of recommendation, contrary to the procedure set out in the call for applications — an inconsistency between the documented design of the process and its technical implementation, introduced by the services responsible for the platform, which was the direct source of the confusion reported by candidates.
This is the second quantitative dimension with the lowest agreement (3.53/5, with 2 responses of disagreement). The final admission results were released within the deadline set out in the call for applications. The source of this perception lies in the results of the procedural review of applications, released two days after the announced deadline — a one-off deviation, with no impact on the final admission decision, but which the Coordination of DocMus.pt records as a point to be corrected in the rigour of compliance with the intermediate deadlines announced.
One candidate noted that ESMAE's application dates are substantially later than those of other institutions (December–February at other programmes, May at ESMAE), which put at risk the candidate's decision in relation to another confirmed offer of admission.
"[I] had to turn down an offer accepted at another institution without knowing whether I would be accepted at ESMAE. This can be detrimental to candidates [...], as well as to the Institution (which may receive fewer applications)."Survey response
Another candidate similarly flagged the competition calendar (phases and deadlines) and the places available and conditions for opening the programme as unclear aspects of the call for applications. As to places, however, the call is explicit: it sets a "maximum of thirteen (13) places" (point 2.1), provides that "places are made available in a single application phase, with a 2nd phase possibly opened should places remain available after Phase 1" (point 2.2), and makes the opening of the programme conditional "on a minimum of six (6) students being duly enrolled" (point 2.4). The Coordination of DocMus.pt notes this perception of a lack of clarity, but does not identify, in the current wording of the call, an objective information gap on this point; it nonetheless recommends visually highlighting these conditions in the text of the next call, given their relevance to the candidate's decision.
This late timing resulted from the fact that A3ES accreditation of the programme was only completed in 2026, with the volume of administrative and preparatory work required to open a new programme in 2026/2027 preventing the application process from being brought forward to a date closer to that used by other institutions. For future editions, with the programme already in consolidated operation, it is recommended that the application process take place during the inter-semester teaching break, in February.
Also recorded, for future reference, are suggestions from candidates that do not share the same degree of convergence among responses, but which were formulated with substantiation. The Coordination of DocMus.pt chose to respond to them individually, rather than deferring them collectively to a future decision, given that they concern options already considered in the design of the programme.
In direct response to the problems identified as priorities (section 6), the Coordination of DocMus.pt commits to implementing the following measures before the opening of the next application process:
| Problem identified | Corrective measure | Responsible / deadline |
|---|---|---|
| Platform architecture subject to multiple valid ways of completing the same requirement (e.g. Ciência Vitae link accepted instead of CV upload in PDF; single field for letters of recommendation) | Redesign the platform with closed forms and a single mandatory format per application requirement, eliminating submission variants not provided for in the call for applications | Academic services / platform management · before the opening of the 2027/2028 call |
| Inconsistent interpretation of the function and expected content of the Digital Artistic Portfolio, despite the call defining its format and criteria | Replace the free-form format with a closed, structured form for the Digital Artistic Portfolio, requiring the information to be presented according to the fields and the curatorial-critical nature intended, and make explicit in the call the rationale for the format's latitude (accommodating materials from different fields) and the admissible-maximum nature of the page limit | Programme Coordination · before the opening of the 2027/2028 call |
| Platform presenting candidates with a field for direct upload of a letter of recommendation, contrary to the direct-submission-by-referee procedure set out in the call | Correct the platform to remove the candidate upload field for letters and ensure that the direct-submission-by-referee flow, with clear instructions, works without failures for up to two letters | Academic services / platform management · before the opening of the 2027/2028 call |
| Release of the procedural review results two days after the deadline set out in the call | Reinforce rigour in complying with the intermediate deadlines announced in the process calendar, including those for releasing the procedural review | Programme Coordination · 2027/2028 call |
| Application dates substantially later than those of other institutions (May, versus December–February), resulting from the completion of A3ES accreditation only in 2026 | Bring the application process forward to the inter-semester teaching break, in February, now that the programme is in consolidated operation | Programme Coordination · 2027/2028 call |
| All respondents (17/17) expressed a wish to receive feedback on their application, with no such procedure yet formalised | Define and communicate a regular feedback procedure for unsuccessful candidates, to be made available after the completion of ranking in each edition | Programme Coordination · 2027/2028 call |
| Application guide prepared for the 1st edition with room for improvement, both in content and in how it was distributed to candidates | Review and strengthen the content of the application guide and ensure its effective distribution to all candidates before the start of the process | Programme Coordination · 2027/2028 call |
The Coordination of DocMus.pt regards this survey as a permanent quality-control instrument for the application process, to be repeated in every edition. The publication of this report aims to guarantee the transparency of the process towards candidates, both future and past, and to document, in a verifiable way, the institutional response to the issues raised.