How do I delete the artboard and only keep the object?

For example - I open a new file in Inkscape (white square art board) and make a circle in the middle. What do I need to do in order to _only_save the picture and get rid of the white background?

If you have not changed the default background, then it is already, automatically transparent. If you Save As SVG, you'll have just the circle in the image. If you use Export Bitmap to get a PNG, it will also be transparent background. But if you used Save As Cairo PNG, it will be a white background.

There might be an exception, if you're on a Mac, and you pasted the circle. That's a known issue, and I can refer you to some tips on using Inkscape on a Mac. Just let me know