Considering that the Linux kernel (the only component Linus has any kind of control over), the software from the GNU Project, and all of the most popular third-party software available for GNU/Linux is all free and open source, it is ridiculous that people can even believe the notion that Linus Torvalds, solely, could put a backdoor in "Linux."
Free and open source software obviously makes a clandestine backdoor very difficult.
There's also the fact that any kind of backdoor in a kernel wouldn't amount to much when most user-space software that initiates kernel actions wouldn't be affected. It's important to remember that Linux can't do most tasks without user-space software, let alone those necessary for a backdoor. For example, the ext4 filesystem libraries and the GNU C Library, as well as the -- often proprietary binary blob -- networking, SATA and video drivers, are all necessary components for a backdoor and would all have to be affected.
Which, again, would not only be a very difficult task, but also one that Torvalds would play a minuscule role in.
I would be more concerned about another possible, more likely, and more viable backdoor vulnerability. That being user-space software. It would be easier to put a backdoor in something like OpenSSL (when it wasn't well maintained), and it would be far more useful to put a backdoor in something like the various filesystem libraries, the GNU C Library, GCC, GIMP, Firefox, or Webkit, as those are used not only on desktop GNU/Linux, but on Android, OS X, BSD, iOS, and Windows as well.
Let's not forget about SELinux, the Linux kernel module developed and maintained by the NSA that sees very wide use across distributions. Since most people already use SELinux, the NSA wouldn't even have to approach Torvalds to get a backdoor into Linux. However, again, SELinux is free and open source, the NSA couldn't put a backdoor in it without people noticing.
All of this is inconsequential, anyway, as Torvalds clarified in September of last year that he was, in fact, joking.
The NSA never actually approached him. Despite what his father says.