System settings → Region and language → Keyboard layout → Advanced → Ctrl key position → Swap control and caps lock
Applications → Settings → Settings Manager → Session and Startup → Application autostart → +Add
/usr/bin/setxkbmap -option "ctrl:swapcaps"
Putting the following in ~/.config/autostart/caps-lock-ctrl-swap.desktop seems to work most of the time:
[Desktop Entry] Encoding=UTF-8 Version=0.9.4 Type=Application Name=setxkbmap Exec=/usr/bin/setxkbmap -option "ctrl:swapcaps" Hidden=false
Debian uses the X keyboard maps for the console too, but configures them differently.