Is everything you see in a small telescope a part of our Milky Way galaxy?

All of the bright stars, star clusters and nebulae are part of the Milky Way. Only the occasional faint smudge of a galaxy such as Andromeda are beyond our Milky Way and are separate systems of stars in the universe.