$ gdb --args ./kfdtest
GNU gdb (Ubuntu 9.2-0ubuntu1~20.04.1) 9.2
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./kfdtest...
(gdb) b KFDLocalMemoryTest.cpp:128
Breakpoint 1 at 0x4bc4e0: file /home/gl/code/ROCm5.2/ROCT-Thunk-Interface/tests/kfdtest/src/KFDLocalMemoryTest.cpp, line 128.
(gdb) r --gtest_filter=KFDLocalMemoryTest.BasicTest
Starting program: /home/gl/code/ROCm5.2/ROCT-Thunk-Interface/tests/kfdtest/build/kfdtest --gtest_filter=KFDLocalMemoryTest.BasicTest
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[ ] Profile: Full Test
[ ] HW capabilities: 0x9
Note: Google Test filter = KFDLocalMemoryTest.BasicTest
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from KFDLocalMemoryTest
[ RUN ] KFDLocalMemoryTest.BasicTest
leiDebug: args.doorbell_offset: 0xd134c00000000000
leiDebug: doorbell_mmap_offset: 0xd134c00000000000
leiDebug: doorbell_offset: 0x0
Breakpoint 1, KFDLocalMemoryTest_BasicTest_Test::TestBody (this=0x555556314b20) at /home/gl/code/ROCm5.2/ROCT-Thunk-Interface/tests/kfdtest/src/KFDLocalMemoryTest.cpp:128
128 dispatch.Sync(g_TestTimeOut);
(gdb)
(gdb) p srcLocalBuffer
$1 = {static Null = {static Null = <same as static member of an already seen type>, m_Flags = {{ui32 = {NonPaged = 0, CachePolicy = 0, ReadOnly = 0, PageSize = 0, HostAccess = 0, NoSubstitute = 0,
GDSMemory = 0, Scratch = 0, AtomicAccessFull = 0, AtomicAccessPartial = 0, ExecuteAccess = 0, CoarseGrain = 0, AQLQueueMemory = 0, FixedAddress = 0, NoNUMABind = 0, Uncached = 0,
Reserved = 0}, Value = 0}}, m_Size = 0, m_pUser = 0x0, m_pBuf = 0x0, m_Local = false, m_Node = 0, m_MappedNodes = 0}, m_Flags = {{ui32 = {NonPaged = 1, CachePolicy = 0, ReadOnly = 0,
PageSize = 0, HostAccess = 0, NoSubstitute = 0, GDSMemory = 0, Scratch = 0, AtomicAccessFull = 0, AtomicAccessPartial = 0, ExecuteAccess = 0, CoarseGrain = 1, AQLQueueMemory = 0,
FixedAddress = 0, NoNUMABind = 0, Uncached = 0, Reserved = 0}, Value = 8193}}, m_Size = 4096, m_pUser = 0x0, m_pBuf = 0x7ffff7fbf000, m_Local = true, m_Node = 1, m_MappedNodes = 2}
(gdb) p 0x7ffff7fbf000
$2 = 140737353871360
(gdb) x/x 0x7ffff7fbf000
0x7ffff7fbf000: 0x01010101
(gdb)
gl@pc:~$ umr -i 1 -vm 0x8@0x7ffff7fbf000 1 -O verbose
=== VM Decoding of address 8@0x7ffff7fbf000 ===
mmVM_CONTEXT8_PAGE_TABLE_START_ADDR_LO32=0x0
mmVM_CONTEXT8_PAGE_TABLE_START_ADDR_HI32=0x0
mmVM_CONTEXT8_PAGE_TABLE_END_ADDR_LO32=0xffffffff
mmVM_CONTEXT8_PAGE_TABLE_END_ADDR_HI32=0xf
mmVM_CONTEXT8_PAGE_TABLE_BASE_ADDR_LO32=0xfebfa001
mmVM_CONTEXT8_PAGE_TABLE_BASE_ADDR_HI32=0x3
mmVM_CONTEXT8_CNTL=0x7ffe07
VMID8.page_table_block_size=0
VMID8.page_table_depth=3
mmVGA_MEMORY_BASE_ADDRESS=0x0
mmVGA_MEMORY_BASE_ADDRESS_HIGH=0x0
mmMC_VM_FB_OFFSET=0x0
mmMC_VM_MX_L1_TLB_CNTL=0x0
mmMC_VM_SYSTEM_APERTURE_LOW_ADDR=0x0
mmMC_VM_SYSTEM_APERTURE_HIGH_ADDR=0x0
mmMC_VM_FB_LOCATION_BASE=0x8000
mmMC_VM_FB_LOCATION_TOP=0x83ff
mmMC_VM_AGP_BASE=0x0
mmMC_VM_AGP_BOT=0x0
mmMC_VM_AGP_TOP=0x0
BASE=0x00000003febfa001, VA=0x7f8000000000, PBA==0x0003febfa000, V=1, S=0, C=0, P=0, FS=0
\-> PDE2@{0x3febfa7f8/ff}=0x00000003fea63001, VA=0x7f8000000000, PBA==0x0003fea63000, V=1, S=0, C=0, P=0, FS=0
\-> PDE1@{0x3fea63ff8/1ff}=0x00000003fea64001, VA=0x007fc0000000, PBA==0x0003fea64000, V=1, S=0, C=0, P=0, FS=0
\-> PDE0@{0x3fea64df8/1bf}=0x00000003fea65001, VA=0x000037e00000, PBA==0x0003fea65000, V=1, S=0, C=0, P=0, FS=0
\-> PTE@{0x3fea65df8/0x1bf}=0x00000003febf8071, VA=0x0000001bf000, PBA==0x0003febf8000, V=1, S=0, C=0, Z=0, X=1, R=1, W=1, FS=0, T=0, MTYPE=NC
\-> Computed address we will read from: vram:3febf8000 (MCA:3febf8000), (reading: 4096 bytes)
=== Completed VM Decoding ===
从上面可以看出，从虚拟地址0x7ffff7fbf000中，前面四级页表的mask分别是，0x7F8、0x7FC、0x37E，0x1BF，最后12bit是页表内offset。借用一张网图：
[sudo] password for gl:
00000000-00000fff : Reserved
00001000-00057fff : System RAM
00058000-00058fff : Reserved
00059000-0009efff : System RAM
0009f000-000fffff : Reserved
000a0000-000bffff : PCI Bus 0000:00
000c0000-000cffff : Video ROM
000f0000-000fffff : System ROM
00100000-a2868017 : System RAM
a2868018-a2878e57 : System RAM
a2878e58-a2879017 : System RAM
a2879018-a2889057 : System RAM
a2889058-a288a017 : System RAM
a288a018-a2898657 : System RAM
a2898658-a2cb8fff : System RAM
a2cb9000-a2cf2fff : ACPI Tables
a2cf3000-a3095fff : System RAM
a3096000-a3096fff : ACPI Non-volatile Storage
a3097000-a3097fff : Reserved
a3098000-ac50afff : System RAM
ac50b000-ac50bfff : Reserved
ac50c000-acbc8fff : System RAM
acbc9000-acc0efff : Reserved
acc0f000-adf29fff : System RAM
adf2a000-adf2afff : Reserved
adf2b000-afe07fff : System RAM
afe08000-b16ebfff : Reserved
b16ec000-b1700fff : ACPI Tables
b1701000-b17fefff : System RAM
b17ff000-b1b26fff : ACPI Non-volatile Storage
b1b27000-b2ffefff : Reserved
b2fff000-b2ffffff : System RAM
b3000000-b7ffffff : Reserved
b4000000-b7ffffff : Graphics Stolen Memory
b8000000-f7ffffff : PCI Bus 0000:00
b8000000-b81fffff : PCI Bus 0000:08
d0000000-dfffffff : 0000:00:02.0
f7b00000-f7cfffff : PCI Bus 0000:01
f7b00000-f7bfffff : PCI Bus 0000:02
f7b00000-f7bfffff : PCI Bus 0000:03
f7b00000-f7b7ffff : 0000:03:00.0
f7b80000-f7b9ffff : 0000:03:00.0
f7ba0000-f7ba3fff : 0000:03:00.1
f7ba0000-f7ba3fff : ICH HD audio
f7c00000-f7c03fff : 0000:01:00.0
f7d00000-f7dfffff : PCI Bus 0000:07
f7d00000-f7d03fff : 0000:07:00.0
f7d00000-f7d03fff : iwlwifi
f7e00000-f7efffff : PCI Bus 0000:06
f7e00000-f7e07fff : 0000:06:00.0
f7e00000-f7e07fff : xhci-hcd
f7f00000-f7f1ffff : 0000:00:1f.6
f7f00000-f7f1ffff : e1000e
f7f20000-f7f23fff : 0000:00:1f.2
f7f24000-f7f25fff : 0000:00:17.0
f7f24000-f7f25fff : ahci
f7f26000-f7f267ff : 0000:00:17.0
f7f26000-f7f267ff : ahci
f7f27000-f7f270ff : 0000:00:17.0
f7f27000-f7f270ff : ahci
f7fe0000-f7ffffff : pnp 00:05
f8000000-fbffffff : PCI MMCONFIG 0000 [bus 00-3f]
f8000000-fbffffff : Reserved
f8000000-fbffffff : pnp 00:05
fd000000-fe7fffff : PCI Bus 0000:00
fd000000-fdabffff : pnp 00:06
fdac0000-fdacffff : pnp 00:06
fdac0000-fdacffff : pnp 00:08
fdad0000-fdadffff : pnp 00:06
fdae0000-fdaeffff : pnp 00:06
fdae0000-fdaeffff : pnp 00:08
fdaf0000-fdafffff : pnp 00:06
fdaf0000-fdafffff : pnp 00:08
fdb00000-fdffffff : pnp 00:06
fdc6000c-fdc6000f : iTCO_wdt
fe000000-fe010fff : Reserved
fe036000-fe03bfff : pnp 00:06
fe03d000-fe3fffff : pnp 00:06
fe410000-fe7fffff : pnp 00:06
fec00000-fec00fff : Reserved
fec00000-fec003ff : IOAPIC 0
fed00000-fed00fff : Reserved
fed00000-fed003ff : HPET 0
fed00000-fed003ff : PNP0103:00
fed10000-fed17fff : pnp 00:05
fed18000-fed18fff : pnp 00:05
fed19000-fed19fff : pnp 00:05
fed20000-fed3ffff : pnp 00:05
fed45000-fed8ffff : pnp 00:05
fed90000-fed90fff : dmar0
fed91000-fed91fff : dmar1
fee00000-fee00fff : Local APIC
fee00000-fee00fff : Reserved
ff000000-ffffffff : Reserved
ff000000-ffffffff : INT0800:00
ff000000-ffffffff : pnp 00:05
100000000-1046ffffff : System RAM
122000000-123002686 : Kernel code
123200000-123c3efff : Kernel rodata
123e00000-12416e13f : Kernel data
124469000-1249fffff : Kernel bss
1047000000-1047ffffff : RAM buffer
2000000000-2fffffffff : PCI Bus 0000:00
2000000000-20001fffff : PCI Bus 0000:08
2200000000-27ffffffff : PCI Bus 0000:01
2200000000-27ffffffff : PCI Bus 0000:02
2200000000-27ffffffff : PCI Bus 0000:03
2200000000-22001fffff : 0000:03:00.0
2400000000-27ffffffff : 0000:03:00.0
2ffe000000-2ffeffffff : 0000:00:02.0
2fff000000-2fff00ffff : 0000:00:1f.3
2fff000000-2fff00ffff : ICH HD audio
2fff010000-2fff01ffff : 0000:00:14.0
2fff010000-2fff01ffff : xhci-hcd
2fff020000-2fff023fff : 0000:00:1f.3
2fff020000-2fff023fff : ICH HD audio
2fff024000-2fff0240ff : 0000:00:1f.4
2fff025000-2fff025fff : 0000:00:16.0
2fff025000-2fff025fff : mei_me
7c00000000-7fffffffff : 0000:03:00.0
gl@pc:~$
gl@pc:~$ sudo devmem2 0x27febf8000
/dev/mem opened.
Memory mapped at address 0x7f88bb971000.
Value at address 0xFEBF8000 (0x7f88bb971000): 0x1010101
gl@pc:~$ umr -i 1 -vm 0x8@0x7ffff5e00000 1 -O verbose
=== VM Decoding of address 8@0x7ffff5e00000 ===
mmVM_CONTEXT8_PAGE_TABLE_START_ADDR_LO32=0x0
mmVM_CONTEXT8_PAGE_TABLE_START_ADDR_HI32=0x0
mmVM_CONTEXT8_PAGE_TABLE_END_ADDR_LO32=0xffffffff
mmVM_CONTEXT8_PAGE_TABLE_END_ADDR_HI32=0xf
mmVM_CONTEXT8_PAGE_TABLE_BASE_ADDR_LO32=0xfebfa001
mmVM_CONTEXT8_PAGE_TABLE_BASE_ADDR_HI32=0x3
mmVM_CONTEXT8_CNTL=0x7ffe07
VMID8.page_table_block_size=0
VMID8.page_table_depth=3
mmVGA_MEMORY_BASE_ADDRESS=0x0
mmVGA_MEMORY_BASE_ADDRESS_HIGH=0x0
mmMC_VM_FB_OFFSET=0x0
mmMC_VM_MX_L1_TLB_CNTL=0x0
mmMC_VM_SYSTEM_APERTURE_LOW_ADDR=0x0
mmMC_VM_SYSTEM_APERTURE_HIGH_ADDR=0x0
mmMC_VM_FB_LOCATION_BASE=0x8000
mmMC_VM_FB_LOCATION_TOP=0x83ff
mmMC_VM_AGP_BASE=0x0
mmMC_VM_AGP_BOT=0x0
mmMC_VM_AGP_TOP=0x0
BASE=0x00000003febfa001, VA=0x7f8000000000, PBA==0x0003febfa000, V=1, S=0, C=0, P=0, FS=0
\-> PDE2@{0x3febfa7f8/ff}=0x00000003fea63001, VA=0x7f8000000000, PBA==0x0003fea63000, V=1, S=0, C=0, P=0, FS=0
\-> PDE1@{0x3fea63ff8/1ff}=0x00000003fea64001, VA=0x007fc0000000, PBA==0x0003fea64000, V=1, S=0, C=0, P=0, FS=0
\-> PDE0-as-PTE@{0x3fea64d78/0x0}=0x00400003fe2004f1, VA=0x000035e00000, PBA==0x0003fe200000, V=1, S=0, C=0, Z=0, X=1, R=1, W=1, FS=9, T=0, MTYPE=NC
\-> Computed address we will read from: vram:3fe200000 (MCA:3fe200000), (reading: 4096 bytes)
=== Completed VM Decoding ===