This article is inspired from Tips & Tricks: Flow Basic Debugging written by @kiwi and I recommend reading that article first before reading this one.
Palo Alto Networks NGFWs use App-ID to detect the exact application inside a traffic stream but sometimes traffic will be first classified for example as App-ID "SSL" and after the decryption it can be classified for example as "web-browsing". This is called application shift and sometimes it can cause issues as when traffic App-ID changes it will go again through the Security Policy Match. If you have a rule that blocks "ssl" at the top and then a rule that allows "web-browsing" then this could be an issue 😉 Also always add the correct parent apps to your custom apps.
For more information see:
How to Prevent Application Shift - Knowledge Base - Palo Alto Networks
How can you see your traffic and the original application App-ID after that. What are the next App-ID's for traffic to see the application shift or if the wrong app-id is matched for some reason? How to see if the final app-id for some reason is not the correct one?
Well this is easy as you just need to enable packet debugging for app-id. For example I used "Example Domain" that is https and it first will be detected as "ssl" and after I configured decryption on the firewall it will be detected as "web-browsing" and you can use a self-generated CA cert as Trusted CA for the SSL emulation as shown in How to Generate a New Self-Signed SSL Certificate - Knowledge Base - Palo Alto Networks.
For testing I suggest using "route ADD" on windows that will cause traffic from your computer to go to the FW ip address and then use "curl" to start traffic. As www.example.com changes it's IP address as it is behind Akamai, use "--resolve" to not do DNS resolution and "-kv" to ignore the errors from the self-signed cert.
route ADD 88.0.0.0 MASK 255.0.0.0 192.168.211.138
curl https://www.example.com/ -kv --resolve www.example.com:443:88.221.211.16
Next in the CLI we enable debugging (I am using VM but note on VM devices you may need to disable session offload "set session offload no"):
debug dataplane packet-diag set filter match source 192.168.211.1 destination 88.221.211.16 destination-port 443
debug dataplane packet-diag set filter match source 88.221.211.16 destination 192.168.211.1 source-port 443
debug dataplane packet-diag set log feature appid all
debug dataplane packet-diag set log on
Packet log is enabled. WARNING: Enabling of debug commands could result in network outage. Not recommended if dataplane CPU is above 60%.
admin@PA-VM> debug dataplane packet-diag show setting
--------------------------------------------------------------------------------
Packet diagnosis setting:
--------------------------------------------------------------------------------
Packet filter
Enabled: yes
Match pre-parsed packet: no
Filter offload: yes
Index 1: 192.168.211.1/32[0]->88.221.211.16/32[443], proto 0
ingress-interface any, egress-interface any, exclude non-IP
Index 2: 88.221.211.16/32[443]->192.168.211.1/32[0], proto 0
ingress-interface any, egress-interface any, exclude non-IP
--------------------------------------------------------------------------------
Logging
Enabled: yes
Log-throttle: no
Sync-log-by-ticks: yes
Features:
appid : basic policy dfa agt
Counters:
Timeout duration: 60 seconds
Buffer threshold: 80%
CPU threshold: 80%
--------------------------------------------------------------------------------
Packet capture
Enabled: no
Snaplen: 0
Username:
--------------------------------------------------------------------------------
After running the curl command and stop loggin you can see the logs (for non VM platforms "less dp-log pan_packet_diag.log" is the command to see logs as they are not in the managment plane but seperate physical data plane):
debug dataplane packet-diag set log off
debug dataplane packet-diag aggregate-logs
less mp-log pan_packet_diag.log
...............................
025-06-19 12:08:02.646 +0300 debug: pan_appid_simple_cache_find(pan_appid_cache.c:518): [CACHE] found app 15 for daddr 0-10d3dd58ffff0000(443, 6) not valid time 5240(5659) count 9(16)
2025-06-19 12:08:02.646 +0300 debug: pan_appid_cache_find(pan_appid_cache.c:725): [CACHE] not found appid 0-1d3a8c0ffff0000(62808) -> 0-10d3dd58ffff0000(62808)
2025-06-19 12:08:02.646 +0300 debug: pan_policy_lookup(pan_policy.c:2302): [ACE] Trigger slow match for appid(0) uappid(0)
2025-06-19 12:08:02.646 +0300 debug: pan_policy_match_service(pan_policy.c:1576): match 0,0 for app 0 uapp 0 proto 6 sport 62808 dport 443
192.168.211.1[62808]-->88.221.211.16[443]
2025-06-19 12:08:02.656 +0300 pan_appid_process_lookup(pan_appid_proc.c:2884): work 0xe03675f100
2025-06-19 12:08:02.656 +0300 debug: pan_session_set_appid_session(pan_appid_priv.h:192): Set appid_session 0xe005fbc100 for session 139
2025-06-19 12:08:02.656 +0300 debug: pan_appid_simple_cache_add_i(pan_appid_cache.c:886): [CACHE] daddr 0-10d3dd58ffff0000(443) appid 15 match appid in cache count 9 threshold 16
2025-06-19 12:08:02.656 +0300 pan_appid_policy_lookup(pan_appid_proc.c:3175): SESSION-DSCP: set session DSCP: 0x00
pan_policy_lookup_fill_arg(normal) : src zone(1) dst zone(2) src(000000000000000000000000c0a8d301:192.168.211.1) src port(62808) dst(00000000000000000000000058ddd310:88.221.211.16) dst port(443) is_ssl(0)
2025-06-19 12:08:02.656 +0300 pan_cfg_app_policy_lookup(pan_cfg_policy.c:1165): Before lookup (short-category-vector): 192.168.211.1[62808]-->88.221.211.16[443]
app 15 use app 15; do lookup 1, app fst 0, url fst 1, rule(1): rule-1
category: any,
session_category(sp): any, any, any, any, any, any
2025-06-19 12:08:02.656 +0300 debug: pan_policy_lookup(pan_policy.c:2302): [ACE] Trigger slow match for appid(15) uappid(0)
2025-06-19 12:08:02.656 +0300 debug: pan_policy_match_service(pan_policy.c:1576): match 0,0 for app 15 uapp 0 proto 6 sport 62808 dport 443
2025-06-19 12:08:02.656 +0300 pan_cfg_set_session_rule_data(pan_cfg_policy.c:1482): SESSION-DSCP: Security policy match, setting session DSCP 0
2025-06-19 12:08:02.656 +0300 pan_cfg_app_policy_lookup(pan_cfg_policy.c:1193): After lookup: 192.168.211.1[62808]-->88.221.211.16[443]
app 15 use app 15; do lookup 1, app fst 0, url fst 1, rule(1): rule-1
category: any,
..............................
192.168.211.1[62808]-->88.221.211.16[443]
2025-06-19 12:08:02.676 +0300 pan_appid_process_match(pan_appid_proc.c:1950): match data 759
2025-06-19 12:08:02.676 +0300 debug: pan_appsig_process_result(pan_app_sigs.c:669): Process app signature [web-browsing] rule [check-get-post-put], dir [cts]
2025-06-19 12:08:02.676 +0300 debug: pan_appsig_process_result(pan_app_sigs.c:695): slot is 1
2025-06-19 12:08:02.676 +0300 debug: pan_appid_check_string_match(pan_app_sigs.c:477): MATCH_ANY: app rule [check-get-post-put] match
2025-06-19 12:08:02.676 +0300 debug: pan_appsig_process_result(pan_app_sigs.c:743): Application found [web-browsing]
2025-06-19 12:08:02.676 +0300 debug: pan_appid_process_match(pan_appid_proc.c:1963): Do policy lookup for session 139 container app: 109
2025-06-19 12:08:02.676 +0300 pan_appid_policy_lookup(pan_appid_proc.c:3175): SESSION-DSCP: set session DSCP: 0x00
pan_policy_lookup_fill_arg(normal) : src zone(1) dst zone(2) s
admin@PA-VM>
The most important parts are "found app 15" and "Do policy lookup for session 139 container app: 109" as this means traffic was first identified as app 15 and then switched to app 109 and thanks to @kiwi the below internal ID can be resolved with the commands below.
admin@PA-VM> debug device-server dump idmgr type shared-application id 15
ssl
admin@PA-VM> debug device-server dump idmgr type shared-application id 109
web-browsing
admin@PA-VM> debug device-server dump idmgr type shared-application name web-browsing
109
admin@PA-VM> debug device-server dump idmgr type shared-application name ssl
15
After that the Palo Alto Networks app wiki at Application Research Center can also be used. Also if there is a wrong App ID you can write your own using the Host header as I have shown in my previous article How to Write Palo Alto Networks Custom Vulnerability and Application Signatures with Examples | Palo...
NOTE: There could be more than one app shifts during a session as for example ssl > web-browsing > gmail-base so keep that in mind!
That's all folks!
