SAP Upgrade Testing Overview
SAP Upgrade Testing Overview
SAP Upgrade-
Simplify R3 September
August Release
Release -
Project ID 110241
EMEA
Testing Deck
June ’22
R3 August Testing Kick off call
April’20
Classification: Internal
Tuesday, March 1
8, 2025
Classification: Internal
Integratio
Upgrade S9D
SPAU
M5D,C5D,B5D CCR
, M9D Retrofit Testing For interface support
D
n
P5D,L5D during Aug and Sep
SPDD
4/14 copy Security Hypercare , testing can
E IT
Validation
remediation happen in N+9 landscape
if needed, but transports
V move in N+5 environment.
Copy SCP,
Integratio
Upgrade S9Q Current
SPAU
MDP,CCP,BWP
D-
, M9Q state
n
P,LTP Initial Releas
SPDD
5/01 prod
Q
D-
Import Ongoing Import TRs es
copy IT
Validation
Sync up
Security
Basis
A Security
setup
Pre data
setup
Master data prep
Securi
prep ty
Dev
PO SIT/Regression
Switch
UAT/Regression
Non- PO
CS Switc Func
Initial h
switch CS Remaining Test
interfaces
R Upgrade MD
Integratio
Copy SCP,
SPAU
P
SRQ
D-
M5P,CCP,
n
BWP,P,LTP MRQ Regr
E +
SPDD
D-
Perf
PO +
R 8/16 prod copy
RPERF Int
F S/ S/
S/M8Q S/MCP
S/ S/ S/
S/MCP
M5D M5Q M9D M9Q MRQ
CURRENT SUPPORT 1909 1909 FUTURE SUPPORT 2021
2021 2021 2021
Classification: Internal
Transpo
Upgrade N+9 Dev Upgrade
SPDD SPAU + Custom code Remediation
Functional adjustments Change Management to provide all Functional adjustments
Functional Validation by IT Tower teams transports from 20th April to 13 May Functional Validation
QA from Prod.
Build N+9 QA, refresh, post refresh basis activities N+9 QA is copy of Prod Build QA
PO Interface as of 30 April S9D, PO Setup
Functional validation by tower IT teams
M9D, C9D, B9D, P9D, Functional Validation
L9D,
Upgrade N+9 QA Upgrade
Delta SPDD Delta SPAU + CCR
Functional Adjustment Functional Adjustments
Data Preparation Master Data Preparation Master Data Preparation
Interface Switch Switch Interface
SIT1 SIT for upgrade
UAT1 UAT for upgrade
Pre-Prod Upgrade
Refresh
Upgrade
Delta SPDD, SPAU
Delta Code remediation
Reg + Perf Testing
S9D S9Q
Refresh Upgrade
Delta SPAU + CCR
Copy of Prod as
of 15 Aug Rreg & Perf
Retrofit Project 1) Breakfix that moves to Prod should be retroffited
SIT
UAT
to to N+9 SRQ
Sept release transports from S5Q
Reg and Perf Testing 2) Parallel testing for Sept release and upgrade 2021
release.
Tuesday, March 1
8, 2025
Classification: Internal
Tuesday, March 1
8, 2025
Classification: Internal
UAT x x
Tuesday, March 1
8, 2025
Classification: Internal
Tuesday, March 1
8, 2025
Classification: Internal
CR handling in ALM
Out of scope:
• Development layer testing
• Project scope testing (not yet in production by September) – this should be driven by the
project teams
Tuesday, March 1
8, 2025
Classification: Internal
CR handling in ALM
Master Data
• Copy of Production as of 4/30/2022
• In-flight projects may need to request setup of Master Data
• Project List? (ITR, Nightfall, Secure Home Delivery,…)
Out of scope:
• Development layer testing
• Project scope testing (not in production by September) – to be driven by the project
teams
Tuesday, March 1
8, 2025
Classification: Internal
• Purpose: Share from each tower their need for collaboration/dependencies from other tower
teams
• Proposed Approach:
• Test captain will check with each tower UAT team if they have any need to have test
help/dependencies from any other tower.
• UAT tower team to prepare the overview on what they need from whom, when
Once we have the feedback ready QA COE team will setup a walkthrough call with the
impacted towers
• Each tower to present their needs
• Discussion to clarify questions, concerns,
• Agree on next steps
Tuesday, March 1
8, 2025
Classification: Internal
Tuesday, March 1
8, 2025
Classification: Internal Signed Off Validation Conditional Not Failed Not started
from In Progress Sign off Applicable
Business Received
E2E Regression Scenarios - APJ
Region HP Biz POC Scenario
702(EOP): Customer Direct ship from In-house Japan Factory – BTO
702(D+): Customer Direct ship from In-house Japan Factory – CTO
JP Kit Mien 007i(D+): Sales Office to Customer from sales office stock (FGCTO)
063a(EOP): Factory Direct Ship to Customer via Physical X-Dock
E2E-007f : Sales Office to Customer (Ship from HP Owned stock)
E2E-002: Factory to DC (Ship to stock / Replenishment) – PS
SEA-K Kit Mien E2E-007: Sales Office to Customer (Ship from HP Owned stock) –B2Bi
E2E-001: IDS / Factory Direct Ship to Customer
E2E-006e: DC to Customer-LSP (Delivery Based 940 Flow) –EOP
E2E-024g: Inv Rebalancing (Intercompany) – DC (loosely coupled) to Sales Office / CDO (loosely
coupled)
AUNZ Kit MIen
E2E-001: IDS / Factory Direct Ship to Customer
E2E-001 AU11
E2E-062: Ship from DC (CTP) to Customer
Region HP Biz POC Scenario
E2E-007f - China
E2E-006 US stock selling to US(US to US) - Delivery based model with standalone intangible
AMS RICARDO RAMIREZ
E2E-002 - Factory to DC (Ship to stock /Replenishment)
E2E-001 – UK Brexit
E2E-001- ODM End state
E2E-063a – Russia RLB
E2E-063e - RLB from RU22 print DIO
19
Classification: Internal
Note: S9Q is a 12 TB environment which may limit the ability to run a full performance test up to 5,00
Tuesday, March 1
8, 2025
Classification: Internal
Tuesday, March 1
8, 2025
Classification: Internal
MD Setup Contacts
I
MD Area / QA COE PoC BIZ PoC for
Elements setup
Material Master Ejaz
W
PLM
Products Costing Ejaz Lizbeth….
Trader Vendors Vinay Kumar Pushpa
Nataraj/Prashant
h/T R Manju
Kumar
P
IC Customers Vinay Kumar Alan Milone/Raji
IC Vendors Vinay Kumar S, Rajesh/Raji
Finance MD Vinay Kumar Monika Jain
Trade Customers Vinay Kumar R, Roopa, Manu
V K,
Logistics Vinay Kumar Alina, Amrin,
Pradeep, Kumar,
Rajesh4
Tuesday, March 1
8, 2025
Classification: Internal
I
30
FI 3
1
…. 2
3
1 3
W
Scenarios ready - 1 Scenarios ready -
3
Identification Identification
2 1
WIP WIP
Setup WIP 30 Setup WIP 6
+0 scenario(s) +0 scenario(s)
ready since ready since
yesterday yesterday
Impact Wave 1
[15 E2E
Wave 2 , TBD
[34 E2E
Delay in MD setup completion for test start 7/25 scenarios] scenarios]
Delay in handover to test team 25
3
11
1
1
5
3
Classification: Internal
BOM 3 / BOM 5 ~ 13 - 15
CTO incl. lower level BOM materials ~ 20 - 25 all BOM requirements need to be submitted within the template and structured
as CTO AV BOMs
CTO only requires extension ~ 13 - 15 PDM will assume extension only if not submitted with lower level BOM
requirements
Trade / IC PIR ~3-5 Can only be requested AFTER material extension & costing is complete
Trade List Price ~ 7 - 10 needs Price Descriptor and needed LP in needed currency
Assumptions
Vendor / Customer & Dependencies
~ 13 - 15
•Logistics
QA COERoute ~ to
involved from project planning 5 -understand
10 relevant scope and required master data
• QA will help to provide overview on existing master data so project team can leverage existing setups instead of creating master
data from scratch where possible
• MD requirements are completely defined and documented using the required templates: Material request template
• New Enterprise Trade Structure requirements (New Sales Orgs, New Plants, New Business Supply Chain flow, etc.) can add another
Classification: Internal
CR Mgmt
Tuesday, March 1
8, 2025
Classification: Internal Overall Status
W
Original
Actions Owner Date Actions Owner New ETA
ETA
Communication Matrix
Test Status Report Status Power point, • Weekly during Frank, Anamika, @ stakeholders,
information on email prep / Vinay OSC
test progress, Thursday
pending steps
and critical open • Daily during
issues/ CRs execution
TBD / if
needed in OSC
Test Team Weekly status & Call Monday / Anamika Test PoC from all
Session(s) - tbd alignment to Wednesday- towers (IT and
ensure Timings- TBD Business) & Test
completion in Captains
time
QA COE Team Alignment within Call Daily Anamika QA COE
catch up team, lessons Timings- TBD
learned, etc.
QA COE War Forum for CR Call Daily –Timings- Ejaz, Vinay CR owners,
Room submitter to seek TBD submitters
help in case
needed
Tower working Daily checkpoint Call Daily Test Captains Tower Teams
sessions and working
session Tuesday, March 1
8, 2025
Classification: Internal
Tuesday, March 1
8, 2025
N+9 QA Landscape
Intranet (SAP Fiori URL) Internet (SAP Fiori URL)
HTTPS HTTPS
SLT
replication
M9Q (S/4HANA
B9Q (BW/4HANA
2021 FP01)
2021 FP01)
EBQ
(1909 FP02)
SPA 7.2 SP10
CHARM Color Legend
C9D (core server N+9 New QA system
Redwood (Scheduler 9.1.3) 2021 FPS01)
Existing system, integrated with N+9
THANK
Simplify R3 August Release -
EMEA YOU
R3 August Testing Kick off call
April’20
ALM Training
• Introduction to ALM
• ALM Overview
• ALM Access
• FAQ
• Common Errors
© Copyright 2016 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Application Lifecycle Management (ALM) Overview
HP Application Lifecycle Management (ALM) is a test management tool which empowers
organizations to manage the core application lifecycle, from requirements through
deployment.
© Copyright 2016 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Classification: Internal
35
Classification: Internal
16. Email the next tester and include the test scenario (e.g. E2E-XXX) and the test
case name and the Test Phase in the SUBJECT of the email. You can also right
37 click any TEST CASE and choose SEND BY EMAIL/SEND BY EMAIL but make
Classification: Internal
• Test Scenarios can be found here in Test Lab
Test Execution: Finding Test • Test Lead will communicate the correct folder you are testing in
(1911 SIT1, 1909 SIT1, 1911 UAT, etc), please make sure you are in
the right folder
Scenarios • Right Click “E2E” and choose EXPAND ALL to see all scenarios
38
Classification: Internal
39
Classification: Internal
Test execution
Test Case Screen
1. You should NEVER update the Status column on the screen. This will create unnecessary
runs that will need to be deleted by ALM Administrator.
2. Status at this level is determined by the pass/fail steps at the STEP LEVEL. Example: If you
have 10 test steps within a test case and all pass, then the status at the TEST CASE level will
show “Passed”. If you have 10 test steps and 9 pass but 1 fails, the TEST CASE level will
show “Failed”. The key is to drill down to the test steps and pass/fail at that level. There are
some inconsistences in ALM regarding how this is reflected.
3. If you really need to change the Status of a test case, get approval from your test tower lead.
It can be done without creating an extra run by opening the test case, choosing RUNS on the
left menu and then updating status on the associated RUN. Test Tower Lead approval is
Classification: Internal
42
Classification: Internal
Attachments: Multiple
screen shots “Copy-
pasted” in word document
Classification: Internal
47 HP Confidential
ALM - Field Level Requirements
48 HP Confidential
Classification: Internal
• MEMO • STATUS
•DATE •PASSED – all steps completed the
automated process successfully
•TESTER’S INITIALS
•FAILED – the results are not as expected, CR
•SO# is created and noted
•PO# •NO RUN – step is not necessary for this
•IC PR# scenario
•N/A – step does not apply to this scenario
•IC PO#
•NOT COMPLETED – tester did not complete
•BILLING DOC# all steps in test case
•INVOICE# •BLOCKED – testing process does not
•FI ACCOUNTING DOC# automatically proceed to the next step
•DATE & Notes •NOT READY – when the Block’s CR identifies
issue preventing the automated process from
continuing to the next step
MEMO - ALWAYS include: date when providing explanation & description of details affecting
the step
49
Classification: Internal
• CR procedure
– Create CR Link: in the test step, create the link between the test step to the CR#
– CR# provided under ACTUAL
51
Classification: Internal
52
Classification: Internal
Role
ID
54 HP Confidential
Classification: Internal
• All defects from the Pre-Integration test phase onwards will be logged and tracked in ALM
• User story and test cases must be linked in ALM so that failing test cases would also fail the
user story
• Identified defects can be technical, functional or non-functional
• All fields of the defect module must be filled with detailed information to allow the assigned
team member to quickly understand the defect and expedite a resolution
56 HP Confidential
Classification: Internal
A defect that brings all test execution to a halt. All testing has
Level 1 – Critical stopped on the sprint. It defers a significant number of test cases for
the cycle because the user cannot perform core functionality. Must
be resolved immediately.
A defect with minimum impact in nature has a work around and does
Level 3 – Medium not prohibit the execution of the test. Does not prohibit tests on the
stated functional area.
Level 4 – Low A defect that is cosmetic in nature and does not prohibit the
execution.
• Defects severity determined by the tester/Defect lead to assure greatest impact items are resolved first so that the
overall test effort can continue
• Following the defect triage with the Defect Lead, Product Owner and the Tower Test Lead, the severity should be
reevaluated
57 HP Confidential
Classification: Internal
Normal Queue. The defect may be fixed within the current drop Work on these issues as soon as there are no
3
or in the next drop longer Priority 1 or 2 in the queue
4 Low Priority. May or may not be fixed at all Work on these last
58 HP Confidential
Classification: Internal
Major system component unusable due to failure or incorrect functionality. < 1 day
Defects cause serious problems such as a lack of functionality, or insufficient or
unclear error messages that can have a major impact to the user, prevents
other areas or towers from being tested, etc.
Level 2 – High
There is a workaround for this defect.
60 HP Confidential
Classification: Internal
• You should always link a Change Request/Defect to
a test step at the TEST STEP LEVEL (not at the
test case level).
CR Linkage To Failed Test Steps
• You have two choices to do this:
• 1. Create a new Defect/CR when you are already
executing. You can do this by choosing “New
Change Request” from the drop down as shown
below. Then enter the data in the fields as shown in
the following slides.
• 2. Link an existing CR to a test step. First you want
to know the CR/Defect #. Then choose “
61 HP Confidential
Classification: Internal
CR Linkage To Failed Test Steps: Creating a CR/Defect from
the Test Run
You should always link a Change Request/Defect to a test step at the TEST STEP LEVEL (not at the test
case level)
You have two choices to do this:
1. Create a new Defect/CR when you are already executing. You can do this by choosing “New Change
Request” from the drop down as shown below. Then enter the data in the fields as shown in the following
slides.
2. Link an existing CR to a test step. See next slide.
62 HP Confidential
CR Linkage To Failed Test Steps: Linking TEST STEP to
Classification: Internal
Existing CR
1. Start a run of a test case to 2.This button will give you an 3. Using the number of the
get to TEST STEP level. Select option to “LINK TO EXISTING CR/Defect, search for the defect
the test step (highlighted in CR”. Click Drop Down to see to create the link.
blue). option.
63 HP Confidential
Classification: Internal
1
Summarize the defect here.
Example: SO52 Incorrect Shipping Point
64 HP Confidential
Classification: Internal
65 HP Confidential
Classification: Internal
66 HP Confidential
Classification: Internal
67 HP Confidential
Classification: Internal
68 HP Confidential
Classification: Internal
69 HP Confidential
Classification: Internal
70 HP Confidential
Classification: Internal
71 HP Confidential
Classification: Internal
Drop Down Owner:
If you know who is the owner search his name
9 and assign
If Not leave it blank
72 HP Confidential
Classification: Internal
73 HP Confidential
Classification: Internal
74 HP Confidential
Classification: Internal
Drop Down Detected in Cycle:
1 Simplify BI is testing in
Releases > Simplify > Year3 > SFY 2102 >
2 SFY 1909, please pick the
SIT1 2102 or UAT 2102
correct cycle in the correct
release
75 HP Confidential
Classification: Internal
76 HP Confidential
Classification: Internal
77 HP Confidential
Classification: Internal
1 1
5 16
4
Status Definition
New A new defect has been created by a tester and assigned to the appropriate team member
Returned The defect logged requires additional information to be triaged, reproduced or resolved
Assigned The defect has been assigned to the team member who will be addressing the defect
The fix team or fixer is actively working on correcting the cause of the defect
In Progress
The defect is ready for the fix team to unit test the fixed defect
FUT Ready
Ready for Retest The defect has been tested successfully by the assignee and is back with the identifier to validate in the testing environment
The defect status is pending the results of a change request and is on hold until the change request is processed through the CR process
Change Request in AgM
Fixed The defect has been fixed and tested by the identifier. The defect is ready for closure by the identifier.
The testing participant has retested the fixed defect and indicates that it has been corrected with proper documentation updated.
Closed
The defect has been reviewed during the CR process and resolution has been deferred to a future date outside of the current cycle. Defect
Deferred status is updated to Deferred. (i.e. A Manual process to be performed until functionality developed in Sprint X)
The defect has been reviewed by the “assigned to” and has been deemed invalid. The person or team rejecting must document the
Rejected reason for rejection and update defect status to Rejected.
This status is selected when a previously closed defect reappears. The defect should again be assigned to the individual responsible for
Reopen fixing it
79 HP Confidential
Classification: Internal
Type Definition
Basis/Infrastructure The defect appears to be caused by a technical issue with the system
Cleansing The defect appears to data cleaning activities performed prior to testing
Design The defect appears to be caused by a flaw in the overall process design
Development - Other The defect appears to be related to a custom development object (RICEFW but not an interface)
Action to be
Root Cause Definition
Taken
True defect as a result of the build of the solution defect type is further
Defect of Build Resolve Defect
elaborated on in the ‘Type of Defect’ field
Missed A requirement that was raised by stakeholders but never formally documented,
Change Request
Requirement designed, or developed during the sprint cycles for a release
New A requirement that was never raised by stakeholders but may be needed for
Change Request
Requirement the final solution
Not a Defect The identified defect is not a true defect Close Defect
81 HP Confidential
ALM - Common Errors
82
Classification: Internal
TROUBLE SHOOTING
Reboot and make sure Global Protect is enabled after reboot. HP Internal users should use Global
Protect at all times by policy. I want to stress this last point for everyone. If you can read this,
you are HP Internal or non-HP Internal, and you should use Global Protect when working
remotely to make sure you have maximum security in your connection.
If you see any errors in IE to load ALM 3, please always try the Client tool:
https://alm-files-and-tools.s3.amazonaws.com/ALM+Tools/ALM3client.zip
Open IE in Private Mode: Ctrl+Shift+P, then access the ALM URL in Private Mode
83
Tuesday, March 1
8, 2025
Classification: Internal
Testing - RACI
QA COE Tower IT Tower Biz Integration TCS
Team
Status A, R C, I C, I C, I
Report/Dashboard
SIT Scope A R C, I C, I
SIT Planning A, R R
SIT Prep & Execution A R
SIT Tracking A R
SIT sign-off A, R R I I
UAT Scope A C, I R
UAT Planning A, R R
UAT Prep & Execution A R
UAT Tracking A R
UAT sign-off A, R R
RT Scope A R
RT Planning A, R R
RT Execution A R
RT Tracking A R
RT Sign-off A, R R
PT Scope, Planning A, R R
PT Execution A Tuesday, March 1
R
8, 2025
Classification: Internal
Task QA- COE Project Enterprise Architects SAP COE TCS- Support
Out of Scope Teams Integration
• Preparation of test Execute A R I C N/A I
cases Functional Testing
• Project Timelines
Coordinate SIT- A R I, C C N/A I
• ALM Readiness - TBD Phase1
• Boundary Systems
Testing Execute SIT A R I C N/A I
Regression A C I, C C I R
Testing
SIT Matrix A, R C I C N/A I
Test Automation- C C A,R C N/A I
Scripts Updates
Test Automation- C I A, R C N/A N/A
Tools
Plan Performance A C C C C I
Test
Execute A C R C C I
Performance
Testing
Creation of test C A,R I I N/A N/A
cases
BOM 3 / BOM 5 ~ 13 - 15
CTO incl. lower level BOM materials ~ 20 - 25 all BOM requirements need to be submitted within the template and structured
as CTO AV BOMs
CTO only requires extension ~ 13 - 15 PDM will assume extension only if not submitted with lower level BOM
requirements
Trade / IC PIR ~3-5 Can only be requested AFTER material extension & costing is complete
Trade List Price ~ 7 - 10 needs Price Descriptor and needed LP in needed currency
Assumptions
Vendor / Customer & Dependencies
~ 13 - 15
•Logistics
QA COERoute ~ to
involved from project planning 5 -understand
10 relevant scope and required master data
• QA will help to provide overview on existing master data so project team can leverage existing setups instead of creating master
data from scratch where possible
• MD requirements are completely defined and documented using the required templates: Material request template
• New Enterprise Trade Structure requirements (New Sales Orgs, New Plants, New Business Supply Chain flow, etc.) can add another
Classification: Internal
88
S8Q Validation Post DB Upgrade
Classification: Internal
Test Preparation:
• Met with Malini to review draft for input on steps and TATs needed
• Sent request to tower teams to confirm PoC for testing phase
• Prepared initial slide deck to address all main topics
• SIT
• UAT
• Regression Testing
• Performance Testing
• Scheduled initial meeting with Tower teams