In-context testing (ICT)

Definition

In-context testing (ICT) is a post-translation process to ensure translation quality by comparing product UI English screenshots with the translated UI screenshots.

During ICT, the tester will check for:

• Correctness: Verify the correctness of translation in context
• Completeness: Verify that there is no translatable data left in English (hardcoded strings)
• Appearance: Verify that the expansion of PII  does not exceeds the space (truncation), Other issues like concatenation, overlap, inappropriate wrapping, alignment, Bidi, etc.
• Consistency: Verify the consistency of translation of titles/labels on different panels.
• Mis-translation: Omissions or mistranslations of keywords or sentences leading to incorrect or incomprehensive translation
• Mismatch: translated version does not match English reference version (back level or wrong version translation)

Workflow

With Globalization Pipeline, product development team can create in-context testing (ICT) translation requests for purpose of translation testing. With ICT translation request, translation testers can perform the testing with attached screenshots in shipment and do the update for resource entries.

Differences from Legacy ICT

• No pre-booking step: There is no need for a Straker TJ number or iteration ID to be submitted with files.
• Single iteration only: The testing process is limited to a single iteration.
• No dependencies on WEP: Screenshots for testing needs to be uploaded to Globalization Pipeline attachments.
• Defect reporting: Translation testers report defects using Globalization pipeline's defect API, with a unique issue tracking ID assigned to each ICT Translation Request.

Steps

The four steps for attachment preparation are:

    Step1: Prepare ICT screenshots

    Step2: Create attachment

    Step3: Upload attachment

    Step4: Associate attachment

    Step5: Tag attachment

After all the attachment are ready, you can proceed to do translation request creation

    Step6: Create a draft translation request

    Step7: Run readiness check and Fix issues

    Step8: Send ICT translation request

Then tester could perform their test with the resource strings in the TR and report issues they found. When the test is done,

    Step9: export the issues translator report to take follow up actions 

Step1: Prepare ICT screenshots

First, you need to prepare the screenshots for translation testing. For in-context testing, you need to prepare the screenshots for source and target languages.

You need to take UI screenshots in one of the following supported image formats, including:

• JPEG
• GIF
• PNG

▸  For more information on preparing screenshots images, refer to " How-to / Prepare UI screenshots".

▸  If you have a huge amount of screenshots and would like to leverage the auto-association feature, you need to prepare the screenshots of pseudo language "English (XP)" as well. Refer to "How-to / Perform auto-association" for more information.

Step2: Create attachment

In order to create screenshot ICT attachments , you can utilize the identical UI or CLI operation as in the ICX flow. For detail information, refer to Process - Translation with context Step2 Create attachment.

Step3: Upload attachment

To upload the screenshots as attachment content, you could follow the same UI or CLI operation as in ICX flow. For detail information, refer to Process - Translation with context Step3 Upload attachment 

Step4: Associate attachment

There are two ways to associate the screenshots to specific resource string: by following the UI or CLI operation within the ICX flow. For a more detailed explanation, please refer to Process - Translation with context Step4 Associate attachment and its guidelines on how to associate attachments.

Step5: Tag attachment

In the In-Context Testing (ICT) flow, you have the flexibility to assign multiple tags to each attachment, allowing you to easily group them into different sets for better organization and management. When generating an ICT translation request, you have the option to define a group of tags that determine the scope of attachments to be included in it With the ability to customize and organize your attachments through tagging, ICT makes it easy to keep track of all your testing materials and minimizing the risk of something being forgotten or overlooked.

Tagging with GP Dashboard

If you would like to assign the "ICT_2023April" tag to identify the ICT shipment to which an attachment belongs, and also specify the UI component for the screenshot, you can easily do so with this workflow.

1. On the Attachments tab, click on the attachment name "quickTranslation-01" .

2. Click on the overflow menu in the header of attachment, and then click on Update attachment

3. Input "ICT_2023April,QuickTrans-UI" in In Attachment tags field, and then click on Update button.

4. You will see the newly created attachment tags in attachment detail.

5. It is also possible to add tags to attachment when you create a new attachment. You could see the attachment tags in creation process. Refer to link for more info in Process - Translation with context Step2 Create attachment section

Tagging with command line interface tool (gp-cli)

To create an attachment with specific tags, you can use the "create-attachment" command. For example, by using this command, you can quickly and easily create an image/jpg attachment with the ID "attachment01" and assign it the tag "ICT_2023April".

The parameters you can specify:

• -a to provide an attachment name that you want to create.
• -t to provide a content type for the screenshot you want to use.
• -g to provide a list of tags separated by comma.
• -j point to a credential file.

The parameters you can specify:

• -a to provide an attachment name that you want to create.

• -g to provide a list of tags separated by comma.

• -j point to a credential file.

The parameters you can specify:

• -al to provide an attachment list name that contains a list of attachment ids you want to create.

• -t to provide a content type for the screenshot you want to use.

• -g to provide a list of tags separated by comma.

• -j point to a credential file.

• For the latest information,

  • refer to the create-attachment command and update-attachment command in gp-cli README page.

Step6: Create a draft translation request

To request an In-Context Testing, you just need to create a draft translation request, which is similar to how you would create a translation request for regular translation or legacy ICT. You are recommended to create the draft translation request for the In-Context Testing in advance because this helps you plan out what you need to do for your testing and make sure that everything is included.

With Dashboard

If you are planning to send an In-Context Testing translation request, for example, in May 2023, it is helpful to create the draft translation request in advance, for example, in April or even earlier.

2.   Pick the bundles you want to include in the translation on step1 and click Next button

3.   Select ICT Request type and input the fields on step3 (see examples below), and then click Next button.

• Expected return time: Normal

  • There are two choices: "Normal" and "Urgent." These options help the translation project manager know how urgent a job is.

• Translation request name: ICT2023April

• Profile ID: default

• Issue Tracking ID: 2023Q1TRdefect

  • Default value for issue tracking id is TR id. It is possible to assign a unique ID to group defects across different translation requests. Assume you want to group all defects in 2023 first quarter into same defect list, you could specify 2023Q1TRdefect as Issue Tracking ID for all the translation requests created in 2023 first quarter.

• Attachment tags: ICT_2023April

  • You can choose to use tags or not. They help to define the scope of the testing attachments.

• Contact email: test@ibm.com

• Special Instructions: ICT_2023April TR instruction

4.  The final step in the process entails reviewing the saved information for your translation request to ensure that all necessary details have been included and that the request is ready for in-context testing. We recommend taking a moment to thoroughly review the information to ensure its accuracy before proceeding.

With command line interface tool (gp-cli)

1. To create draft ICT translation request, you can use "submit-ict-tr" command to create a draft translation request.

The parameters you can specify:

• -j point to a credential file.
• -g to provide a comma separated list of attachment tags to be associated with this TR.
• -t to provide translation note that you want to add.
• -l to provide a comma separated target language list.
• -e to provide additional e-mail addresses for status notification
• -b to provide a comma separated bundle list.
• -n to provide a translation request name.

Step7: Run readiness check and Fix issues

After completing the preparation of the screenshot and creating the draft TR, the next step is to perform a readiness check to confirm that the translation request is ready for submission.

With Dashboard

1. By clicking on "Check readiness" during the final step of creating a translation request, you can initiate the readiness check process right away.

2. Click OK button to start the readiness check.

3. The readiness check process may take some time to complete. Once you receive a notification email, you can return to the dashboard to view the results. If the Readiness check status displays as "Ready", it indicates that there are no critical issues with the request and that the translation request is now suitable for submission.

4. To ensure that you have not missed any test items by mistake, it is advisable to review the readiness check details. You can view the readiness check results by clicking on the name of the translation request. Additionally, you can download the readiness check report in CSV format, which can be useful for filtering, sorting, or keeping it as a log.

5. To obtain more information regarding the validation items and how to resolve any errors identified during the readiness check process, please refer to the "How to - Readiness Check" guide.

6. Once you have followed the instructions provided to address the errors identified in the report, you can proceed to run the readiness check once again.

7. Continue to repeat the previous two steps until the translation is completely error-free. At this point, the translation request will be ready for submission.

With command line interface tool (gp-cli)

1. If you wish to review the content of a draft ICT translation request, you can utilize the "check-ict-tr" command to perform a readiness check on the request.

The parameters you can specify:

• -t to provide the translation request id that you want to submit.
• --items to specify the type of check you wish to perform, for ICT shipment checks, you can use "ict-content"
• -j point to a credential file.

The parameters you can specify:

• -t to provide the translation request id that you want to get.

• -o to specify exported file name

• -j point to a credential file.

Step8: Send ICT translation request

Once the readiness check is completed without any errors, the readiness status of the translation request will be updated to "READY", indicating that it is now suitable for submission. You may proceed to send the translation request at this point. If you change the ICT scope or updated the resource entries, updated translation values, attachments or attachment contents and tags for this ICT translation request, you will need to re-run the readiness check before submission.

With Dashboard

1.   On the Translation Requests tab, click on the overflow menu and click on Submit.

With command line interface tool (gp-cli)

1.   To submit a translation request that is in the "Ready" status, utilize the "submit-ict-tr" command.

The parameters you can specify:

• -t to provide the translation request id that you want to submit.
• -j point to a credential file.

Step9: Review issues report

When the testing is done, you could export the defect list from dashboard or cli to review them and take follow up actions.

With Dashboard

1. On the Translation Requests tab, click on the translation request name, or click on overflow menu to select View the translation request detail.

2. To view a summary of the defects identified in the translation request, scroll to the bottom of the translation request detail page. Here, you will find a table that summarizes the number of defects present in each language and their respective severity levels.

3. To obtain further details regarding the defects identified during the In-Context testing , you can download the zip file for more comprehensive information or the CSV file for a quick summary.

With command line interface tool (gp-cli)

1. Utilize the "export-defects" command to export all defects reported by translation testers as an attachment, using a specified tracking ID.

The parameters you can specify:

• -t to provide Issue Tracking ID
• -r to specify the report type. Currently HTML or CSV report type could be generated
• -o to specify the file name where the exported data will be written.
• -j point to a credential file.