MIT 6.S081 Lab4: Traps

Posted on Edited on In
  • 跟着视频走一遍系统调用gdb的流程。tmux分割两个窗口，一个窗口作为服务器make CPUS=1 qemu-gdb，另一个窗口作为gdb调试窗口gdb-multiarch。将断点打在ecall指令处, continue执行，随后再将第二个断点打到print/x $stvec处也就是，TRAPFRAME的起始地址。ecall指令完成三件事，将用户模式切换到管理员模式、将PC保存到sepc寄存器中、将stvec寄存器的值赋给PC跳转到stvec保存的地址处执行。
  • 进入trampoline.s后，csrrw a0, sscratch, a0首先将非体系结构寄存器sscratch与a0的值交换，sscratch寄存器中保存的时TRAPFRAME的起始地址。
  • 然后将当前的现场(即寄存器)保存到TRAPFRAME中，再将TRAPFRAME中保存的内核栈指针，hartid，usertrap()的地址，以及内核页表所在的stap寄存器的值加载到当前通用寄存器中。
  • 随后将寄存器t1保存的内核页目录的地址写入当前satp寄存器中，再刷新TLB，将用户页表切换为内核页表 
    1
    2
    csrw satp, t1
    sfence.vma zero, zero
  • 将用户模式完全切换为内核模式之后，最后跳到t0中保存的usertrap()的入口地址，jr t0跳入usertrap函数。
  • 进入函数后，首先判断sstatus寄存器的SSP位是否为0(即是否为用户模式下发生的trap)。将stvec赋值为kernelvec的入口地址，即在当前usertrap发生的中断或异常则跳转到stvec处执行。再保存中断返回的地址到sepc。
  • 中断将会改变sstatus寄存器，因此在修改结束之后才将中断打开intr_on()。
  • 之后进入系统调用syscall()，根据p->trapframe->a7中保存的系统调用号来决定调用哪个系统调用(专门通过一个静态数组查找syscalls[num]), 随后将系统调用的返回值保存到p->trapframe->a0中。
  • 完成系统调用之后，随后计算uservec的虚拟地址并赋值给stvec，以便发生异常或中断的时候处理。接下来将相应的内容restore到trapframe中方便进行下一次trap。随后更新sstatus状态寄存器的值，清空SSP、设置SPIE位。
  • 更新sepc的值以及将satp的值设为内核页表的地址，将在userret中切换页表。计算userret在trampoline.s中的虚拟地址，跳转到userret，跳转之前传参有个小细节，即将TRAMPOLINE作为第一个参数，这样在a0与sscratch交换后，sscratch就得到TRAMPOLINE的起始地址了。
  • 进入到收尾阶段，将恢复到trap之前的状态。将TRAMPOLINE中的内容load到通用寄存器中，先将a0寄存器的值写入sscratch寄存器中，这样最后csrrw a0, sscratch, a0即可将这两个寄存器复位为各自的值。最后sret将sepc赋给pc完成系统调用恢复正常执行。

比较重要的非体系结构寄存器

  • stvec, 存放系统调用处理程序的地址
  • sepc, 当系统调用发生时PC存放到此处，以便系统调用返回时能从下一条指令开始执行sret: sepc -> pc。
  • scause, ISA通过它来分析系统调用的种类
  • sscratch, 内核将一个值放到这里，方便系统调用的开始(通用寄存器和sscratch寄存器通过csrrw来交换值csrrw a0, sscratch, a0)。
  • sstatus, 状态寄存器，类似LC-3来决定是否发生中断，或者决定是用户模式还是系统模式，若存在条件码则存放条件码。

Debug相关

  • add-symbol-file或file命令从文件filename中读取附加的符号表信息存放在ELF文件(可重定向目标文件)中的.symtab Entry中。当文件名（通过其他方式）动态加载到正在运行的程序中时，将使用此命令。
  • 解决调试alarmtest时usertrap C源代码不显示的, 函数和变量信息不够全，需要add-symbol-file kernel/kernel即可。

1. RISC-V assembly

1.1 Description

It will be important to understand a bit of RISC-V assembly, which you were exposed to in 6.004. There is a file user/call.c in your xv6 repo. make fs.img compiles it and also produces a readable assembly version of the program in user/call.asm.

可以在gdb中使用file来对call.o文件调试，并将断点打到main函数上。解释调试时RISCV汇编出现的一些指令, x表示寄存器, M表示存储器:

  • auipc, Add Upper Immediate to PC. 将指令编码格式中的Imm[31:12]左移12位后的结果sign-extened后再加上PC。
    1
    auipc rd, Imm  ; x[rd] = PC + sext(Imm[31:12] << 12)
  • li(pseudoinstruction), Load Immediate.
    1
    li rd, Imm  ; x[rd] = Imm
  • mv(pseudoinstruction), Move. 注意与x86 ISA的mov传递方向不同。
    1
    mv rd, rs1  ; x[rd] = x[rs1]
  • jalr, Jump And Link Register. 为什么要将最低有效位置为0?字节对齐。将当前pc+4赋给ra作为返回地址并跳转到offset(rs1), 随后返回到当前指令的下一条指令继续执行。注意如果rd省略了，那么rd就默认为x1(即ra保存返回地址的寄存器)。
    1
    jalr rd, offset(rs1)  ; t=pc+4; pc=(x[rs1]+sext(offset)) & ~1; x[rd] = t;
  • lbu, Load Byte Unsigned. 取完一个字节后，紧接着零拓展, lb为符号位拓展。
    1
    lbu rd, offset(rs1)  ; x[rd] M[x[rs1] + sext(offset)] [7:0]
  • seqz(pseudoinstruction), Set if Equal to Zero.
    1
    seqz rd, rs1  ; x[rd] = (x[rs1] == 0)
  • csrrw, Control and Status Register Read and Write. 状态寄存器和通用寄存器之间的读写操作。将状态寄存器中的内容放入rd寄存器，将rs1的内容放入状态寄存器。
    1
    csrrw rd, csr, rs1  ; t = CSRs[csr]; CSRs[csr] = x[rs1]; x[rd] = t
  • csrw
    1
    csrw csr, rs1
  • csrr
    1
    csrr rd, csr

    1.2 Implementation

  • 不了解RISCV指令集的建议可以把CS61C的Week2专门讲RISCV的slide或者视频看完，直到把调试过程中遇到的每条指令弄明白再来做这一个task。

分析user/call.asm

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
void main(void) {
  1c:	1141                	addi	sp,sp,-16
  1e:	e406                	sd	ra,8(sp)
  20:	e022                	sd	s0,0(sp)
  22:	0800                	addi	s0,sp,16
  printf("%d %d\n", f(8)+1, 13);
  24:	4635                	li	a2,13
  26:	45b1                	li	a1,12
  28:	00000517          	auipc	a0,0x0
  2c:	7c050513          	addi	a0,a0,1984 # 7e8 <malloc+0xea>
  30:	00000097          	auipc	ra,0x0
  34:	610080e7          	jalr	1552(ra) # 640 <printf>
  exit(0);
  38:	4501                	li	a0,0
  3a:	00000097          	auipc	ra,0x0
  3e:	27e080e7          	jalr	638(ra) # 2b8 <exit>
  • Which registers contain arguments to functions? For example, which register holds 13 in main’s call to printf?
    1
    a2
  • Where is the call to function f in the assembly code for main? Where is the call to g? (Hint: the compiler may inline funtions.)
    1
    编译器优化: g被inline到了f中，f又进一步被内联到了main中
  • At what address is the function printf located?
    1
    auipc ra, 0x0指令将0x30赋给ra中, 而jalr 1552(ra)跳转到的地址为0x640
  • What value is in the register ra just after the jalr to printf in main?
    1
    如果jalr没有第一个操作数，那么返回地址默认存放到ra寄存器中。ra=pc+4即0x38
  • Run the following code.
    1
    2
    3
    4
    unsigned int i = 0x00646c72;
    printf("H%x Wo%s", 57616, &i);
     ```     
     What is the output?
    57616=0xe110, RISCV是little-endian, 因此i在内存中存储的形式为0x726c6400对应的ASCII值为0x72 = ‘r’, 0x6c = ‘l’, 0x64 = ‘d’, 0x00 = ‘\0’。因此最后printf输出的结果为”He110, World\0”。
    1
    If the RISC-V were instead big-endian what would you set i to in order to yield the same output?
    如果是大端字节序，为保持相同的输出结果，将i的值反过来即可i = 0x726c6400
    1
    2
    3
    - In the following code, what is going to be printed after 'y='? (note: the answer is not a specific value.) Why does this happen?
      ``` c
    	printf("x=%d y=%d", 3);
    1
    阅读call.asm, 将3存放到a1寄存器后，在调用printf之前并未对a2寄存器进行修改(本应该有第二个参数的), 第二个参数传入的值是a2寄存器中原有的随机值。

2. Backtrace

2.1 Description

在kernel/printf.c中实现backtrace()函数。阅读源码时会有内联汇编的相关知识。实现backtrace函数还需要了解RISCV栈帧布局。当前的stack frame含有对前一个stack frame的指针。高地址往低地址以此为Return Address, To Prev. Frame Pointer, Saved Registers, Local Variables…其中frame pointer指向栈帧第一个entry的顶部。
Some hints

  • Add the prototype for backtrace to kernel/defs.h so that you can invoke backtrace in sys_sleep.
  • The GCC compiler stores the frame pointer of the currently executing function in the register s0. Add the following function to kernel/riscv.h:
    1
    2
    3
    4
    5
    6
    7
    static inline uint64
    r_fp()
    {
      uint64 x;
      asm volatile("mv %0, s0" : "=r" (x) );
      return x;
    }
    and call this function in backtrace to read the current frame pointer. This function uses in-line assembly to read s0.
  • These lecture notes have a picture of the layout of stack frames. Note that the return address lives at a fixed offset (-8) from the frame pointer of a stackframe, and that the saved frame pointer lives at fixed offset (-16) from the frame pointer.
  • Xv6 allocates one page for each stack in the xv6 kernel at PAGE-aligned address. You can compute the top and bottom address of the stack page by using PGROUNDDOWN(fp) and PGROUNDUP(fp) (see kernel/riscv.h. These number are helpful for backtrace to terminate its loop.

    2.2 Implementation

    要利用好当前栈帧所在页的边界，来通过每个栈帧的prev(类似于链表)，来遍历并打印当前栈帧的返回地址。要注意栈是由高地址向低地址方向增长的，因此需要获取页的Top作为边界, 要理解回溯这个词。注意更新fp的时候上一个栈帧的fp是存放在地址单元为当前栈帧的fp-8中的, 因此需要解引用(*)取地址。在xv6中，内核为进程分配一个页大小的栈。
    kernel/printf.c
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    void
    backtrace(void) {
      uint64 fp = r_fp();  // r_fp() return the fp of current execute function
      uint64 ftop = PGROUNDUP(fp);  // get the top addr of stack frame page.
      printf("backtrace:\n");
      while (fp < ftop) {
        printf("%p\n", *(uint64*)(fp-8));  // print return address stored in (fp-8).
        fp = *(uint64*)(fp-16);   //  update fp to previous frame fp.
      }
    }

3. Alarm

3.1 Description

In this exercise you’ll add a feature to xv6 that periodically alerts a process as it uses CPU time. This might be useful for compute-bound processes that want to limit how much CPU time they chew up, or for processes that want to compute but also want to take some periodic action. More generally, you’ll be implementing a primitive form of user-level interrupt/fault handlers; you could use something similar to handle page faults in the application, for example

sigalarm(n, fn)表示，在每个n ticks之后将会调用应用程序函数fn, 当fn函数调用结束之后，应用程序将会在它调用fn的地址处恢复执行。

3.2.1 test0: invoke handler

Some hints:

  • You’ll need to modify the Makefile to cause alarmtest.c to be compiled as an xv6 user program.
  • The right declarations to put in user/user.h are: 
    1
    2
    int sigalarm(int ticks, void (*handler)());
    int sigreturn(void);
  • Update user/usys.pl (which generates user/usys.S), kernel/syscall.h, and kernel/syscall.c to allow alarmtest to invoke the sigalarm and sigreturn system calls.
  • For now, your sys_sigreturn should just return zero.
  • Your sys_sigalarm() should store the alarm interval and the pointer to the handler function in new fields in the proc structure (in kernel/proc.h).
  • You’ll need to keep track of how many ticks have passed since the last call (or are left until the next call) to a process’s alarm handler; you’ll need a new field in struct proc for this too. You can initialize proc fields in allocproc() in proc.c.
  • Every tick, the hardware clock forces an interrupt, which is handled in usertrap() in kernel/trap.c.
  • You only want to manipulate a process’s alarm ticks if there’s a timer interrupt; you want something like 
    1
    if(which_dev == 2) ...
  • Only invoke the alarm function if the process has a timer outstanding. Note that the address of the user’s alarm function might be 0 (e.g., in user/alarmtest.asm, periodic is at address 0).
  • You’ll need to modify usertrap() so that when a process’s alarm interval expires, the user process executes the handler function. When a trap on the RISC-V returns to user space, what determines the instruction address at which user-space code resumes execution?
  • It will be easier to look at traps with gdb if you tell qemu to use only one CPU, which you can do by running 
    1
    make CPUS=1 qemu-gdb
  • You’ve succeeded if alarmtest prints “alarm!”.

    3.2.2 test0 implementation

  • kernel/proc.h，在proc结构体中加入结构体成员tick, handler, intervel。
    1
    2
    3
    uint64 tick;
    uint64 handler;
    uint64 intervel;
  • kernel/proc.c，在allocproc函数中初始化tick成员为0。
    1
    p->tick = 0;
  • kernel/trap.c, 当trap返回时，sret指令将sepc寄存器中的地址赋给pc执行，也就是说在下述代码中，trap返回时将执行alarm的中断处理程序。
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    // give up the CPU if this is a timer interrupt.
    if(which_dev == 2) {
      if (p->intervel == p->tick) {  // expire
        p->tick = 0;  // reset tick
        // save all the needed registers
        // epc store the user program counter(PC).
        p->trapframe->epc = p->handler;  // when returned, jump to execute handler.
      } else {
        p->tick++;
      }
      yield();
    }
  • kernel/sysproc.c, argint和argaddr获取系统调用的第n个参数值, 然后初始化进程中对应的属性。
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    uint64
    sys_sigalarm(void) {
      struct proc* p = myproc();
      int interval;
      uint64 handler;
      // fetch syscall nth argument.
      if (argint(0, &interval) < 0)
        return -1;
      if (argaddr(1, &handler) < 0)
        return -1;
      // initialize p's attribute.
      p->intervel = interval;
      p->handler = handler;
      return 0;
    }

    3.3.1 test1/test2: resume interrupted code

    需要添加一些操作，确保在alarm处理程序完成后，控制权返回到用户程序最初被时钟中断的指令。必须得确保寄存器内容恢复到中断前的值，以及中断前的位置，所以需要在alarm的handler覆盖掉sepc之前保存好存放返回地址的sepc寄存器。
    Some hints:
  • Your solution will require you to save and restore registers—what registers do you need to save and restore to resume the interrupted code correctly? (Hint: it will be many).
  • Have usertrap save enough state in struct proc when the timer goes off that sigreturn can correctly return to the interrupted user code.
  • Prevent re-entrant calls to the handler—-if a handler hasn’t returned yet, the kernel shouldn’t call it again. test2 tests this.

3.3.2 test1/test2 Implementation

考虑一下第一个提示，是不是可以选择性地save寄存器? 看一下alarmtest.asm中的handler的汇编程序找一找。

  • 在kernel/proc.h中加入is_alarm_working属性，防止当前还在执行handler而导致的重入。
    1
    int is_alarm_working = 0;
  • kernel/trap.c, 因为为用户级别的中断，因此不涉及到trapframe中保存的有关内核寄存器的修改。
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    // give up the CPU if this is a timer interrupt.
    if(which_dev == 2) {
      if (!p->is_alarm_working && p->intervel > 0) {
        if (p->intervel == p->tick) {  // expire
          p->tick = 0;  // reset tick
          p->is_alarm_working = 1;  // reprensent executed handler is not terminate.
          // save all the needed registers.
          p->saved_epc = p->trapframe->epc; // save return address.
          p->saved_ra = p->trapframe->ra;
          p->saved_sp = p->trapframe->sp;
          p->saved_gp = p->trapframe->gp;
          p->saved_tp = p->trapframe->tp;
          p->saved_t0 = p->trapframe->t0;
          p->saved_t1 = p->trapframe->t1;
          p->saved_t2 = p->trapframe->t2;
          p->saved_t3 = p->trapframe->t3;
          p->saved_t4 = p->trapframe->t4;
          p->saved_t5 = p->trapframe->t5;
          p->saved_t6 = p->trapframe->t6;
          p->saved_a0 = p->trapframe->a0;
          p->saved_a1 = p->trapframe->a1;
          p->saved_a2 = p->trapframe->a2;
          p->saved_a3 = p->trapframe->a3;
          p->saved_a4 = p->trapframe->a4;
          p->saved_a5 = p->trapframe->a5;
          p->saved_a6 = p->trapframe->a6;
          p->saved_a7 = p->trapframe->a7;
          p->saved_s0 = p->trapframe->s0;
          p->saved_s1 = p->trapframe->s1;
          p->saved_s2 = p->trapframe->s2;
          p->saved_s3 = p->trapframe->s3;
          p->saved_s4 = p->trapframe->s4;
          p->saved_s5 = p->trapframe->s5;
          p->saved_s6 = p->trapframe->s6;
          p->saved_s7 = p->trapframe->s7;
          p->saved_s8 = p->trapframe->s8;
          p->saved_s9 = p->trapframe->s9;
          p->saved_s10 = p->trapframe->s10;
          p->saved_s11 = p->trapframe->s11;
          // epc register store the user program counter(PC).
          p->trapframe->epc = p->handler;  // when returned, jump to execute handler.
        } else {
          p->tick++;
        }
      }
      yield();
    }
  • kernel/sysproc.c, sigreturn系统调用在alarm的handler结束之后完成sigalarm发生前现场的保护，即对寄存器的restore和防止重入变量的reset。
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    uint64
    sys_sigreturn(void) {
      struct proc* p = myproc();
      p->trapframe->epc = p->saved_epc; 
      p->trapframe->ra = p->saved_ra; 
      p->trapframe->sp = p->saved_sp; 
      p->trapframe->gp = p->saved_gp; 
      p->trapframe->tp = p->saved_tp; 
      p->trapframe->a0 = p->saved_a0; 
      p->trapframe->a1 = p->saved_a1; 
      p->trapframe->a2 = p->saved_a2; 
      p->trapframe->a3 = p->saved_a3; 
      p->trapframe->a4 = p->saved_a4; 
      p->trapframe->a5 = p->saved_a5; 
      p->trapframe->a6 = p->saved_a6; 
      p->trapframe->a7 = p->saved_a7; 
      p->trapframe->t0 = p->saved_t0; 
      p->trapframe->t1 = p->saved_t1; 
      p->trapframe->t2 = p->saved_t2; 
      p->trapframe->t3 = p->saved_t3; 
      p->trapframe->t4 = p->saved_t4; 
      p->trapframe->t5 = p->saved_t5; 
      p->trapframe->t6 = p->saved_t6;
      p->trapframe->s0 = p->saved_s0;
      p->trapframe->s1 = p->saved_s1;
      p->trapframe->s2 = p->saved_s2;
      p->trapframe->s3 = p->saved_s3;
      p->trapframe->s4 = p->saved_s4;
      p->trapframe->s5 = p->saved_s5;
      p->trapframe->s6 = p->saved_s6;
      p->trapframe->s7 = p->saved_s7;
      p->trapframe->s8 = p->saved_s8;
      p->trapframe->s9 = p->saved_s9;
      p->trapframe->s10 = p->saved_s10;
      p->trapframe->s11 = p->saved_s11;
      p->is_alarm_working = 0;
      return 0;
    }

最后的疑问

用户级别的时钟中断是在系统调用sigalarm时对硬件进行操作使得其周期性地发生中断?

All Test Passed

Test

总结

终于结束lab4了，呼~, gdb调的真舒服，基本的syscall逻辑大致都搞明白了。Backtrace按照hint写入一些函数，实际上就是要理解上一节video讲的frame point的一些概念，当前stack frame的第1个entry会指向前一个frame同时也给出了stack frame的分布图，间接地实现了gdb的查看栈帧的backtrace命令; 系统调用alarm会在进程使用CPU时间定期发出警报，手把手实现一个用户级别的中断。通过trap中根据devintr()的返回值判断中断的类型，1为设备中断，2为时钟中断，0为未识别。进而tick控制在指定intervel内调用中断处理程序handler。同时还需要在proc.h中加入字段，防止中断发生时的重入。涉及到部分寄存器的store/restore。另外ecall指令完成三件事：1.将用户模式切换到管理员模式; 2.将PC保存到sepc寄存器中; 3.将跳转到stvec寄存器中存储的地址处执行，即将stvec赋给PC。