How are you installing Linux?
Do you keep the Windows partition? Or did you let the installer you tried overwrite everything on the hard disk?
When you do the latter, it makes installing Linux much simpler.
But when you have or want a dual boot Atom machine, chances are that the bootloader (GRUB most likely) is getting overwritten during your reboot procedure.
You could try
this to attempt to repair the boot sector(s) from the hard drive in your Atom machine. Looks to me one of the easiest boot repair tools to use. Once the boot sector(s) are set correctly, the GRUB bootloader can find the boot files it needs in the location where it expects these files to be. Once it does you should get a very basic screen with several boot options. By default the most likely boot option you need is already pre-selected and you can either wait a few seconds or hit the 'Enter' key to continue.
If you fancy a dual boot machine, better install Windows first, then Linux. The GRUB bootloader is able to cope with Windows. The Windows bootloader is (and likely never will be) capable to "see" an existing Linux installation. Dual boot is not what I would recommend though.
Is the Atom machine easy to open? If it is, I would exchange the current hard disk with a fresh SSD drive. A 120 GByte model should not be too expensive. That way you would not lose anything you have stored on that machine and you have a fresh drive to start and play with Linux. With an easy to open case you could exchange drives when needed. May be the best way to go about it anyway, as you won't lose any data accidentally. Within Linux it is not a problem to read files/folders from a NTFS (or FAT32) partition, but where things may go awry is writing to such a partition. Especially with stripped down versions of Linux like Puppy Linux, Tiny Linux, Porteus, etc.
For example: next year my router PC will be 15 years old. In my network there are 3 more computers that are 10+ years old. One of these is still a Pentium 4 class PC. All stuff I want to backup from other computers (Windows/Linux) in the network are stored on this Pentium 4 class PC by each client PC. The only thing the P4 machine does is creating archives from the collected files and fill backup files with these archives using Bacula. The P4 machine has a GUI-less Linux on it and the system is managed by through a web interface called: webmin. 5 to 6 hours it is busy creating archives, the rest of the time it is receiving files to backup and generating (graphical) reports from the backups, which can be accessed through another web-interface. If it wasn't for the pending drop of 32-bit support in Ubuntu Server editions, this little machine would still function just fine.
The example above hopefully shows that we share a similar mindset regarding old hardware. Re-purpose is the name of that game.
Not to offend, but the suggestion of my previous post: get a slightly better laptop and try a full-fledged version of Linux on that, still stands. All those small Linux distros have one thing in common, compromises in software and functionality in one way or another. Those compromises are likely to bite you in the end, unlike a full version of Linux.
My previous post also suggested to wait with Linux on the Atom machine until you feel confident enough that you can re-purpose it for tasks running on a Server edition of Linux without GUI, which is so much easier on the available resources in your Atom machine. It could become your own personal DNS server, for example. Or let it monitor all traffic in your network. Perhaps use it to manage 2 to 3 (lower resolution) security cameras for your house. Or other similar tasks you deem useful.
