Keep a flat, fine-toothed file handy for on-the-job scraper blade sharpening, or for filing down small nicks on the blade. Depending on the style of double-edged scraper you have, you may need to unscrew the blade and remove it from the scraper handle first.
Lay the blade flat on a workbench or other flat surface. Hold one side of the blade flat. Run the file lengthwise along the edge. Turn the blade around and file the other side. Flip the blade over and repeat.
Sharpen the scraper blade more finely with a whetstone -- a sharpening stone -- after you've filed the blade. Wet-dry sandpaper works as well. Put a few drops of mineral oil on the blade before you sharpen it for a very smooth finish.
Use an electric rotary grinder for efficiency if you have a lot of blades to sharpen.