From the north shore of Long Island, New York between 9:30PM and 3:30 AM I keep seeing stars in the eastern sky that move and zoom around in loop-de-loops, always returning to the same spot. What's going on?

From your location, looking eastwards is a line of sight that goes along the coast and crosses part of the eastern tip of Long Island. There are many airfields on Long Island, and also military airfields such as Suffolk County Air Force base near Eastport. If you say the lights between 9:30PM and 3:30 AM, the sky rotates 1/4 turn in this 6 hour period so that objects seen by the horizon at 9:30PM would be straight overhead by 3:30AM and since you did not mention this very obvious 'spectacular' effect, it could not be stars you are seeing. I hate to say this, but considering that there are good sources of aviation traffic along your eastern view, that is my best guess.