This is a complicated process as you can well imagine. AS an astronomer, this is the way I understand how the process works for NASA.
The astronomical community advises the NASA Astrophysics Division about which scientific goals it considers the highest priority. Since 1970, this has happened through a series of National Science Foundation 'Commissions' on Astrophysics in the 1970's, the 1980's and currently the 1990's. The most recent of these is called the 'Bahcall Report' after Prof John Bahcall, the Chairman of the committee that was empaneled in the late 1980's to study and prioritize the various projects and programs that the astronomical community thought should be undertaken in the 1990's. This report gives an overall theme to the directions of astrophysical research in this decade, and highlights those questions that are ready for aggressive investigation using new space-based resources, observatories, spacecraft etc. The inputs to this report were requested of the entire astronomical community by the Bahcall Commission, which then reviewed all of the suggestions, and created from them a cohesive plan of investigation.
The Astrophysics Division at NASA Headquarters uses the Bahcall Report recommendations to plan new-starts, and already-funded programs. The Division creates a budget that includes the new-starts assigns them an acronym. If Congress accepts the request for new funding, or agrees with whatever diversion of resources NASA proposes to fund the new research programs, then the program enters the next phase. NASA issues an RFP or 'Request for Proposals' to the astronomical community for astronomers to propose what the new mission ought to look like, what instruments it ought to carry, and all of the optical, sensor, mechanical characteristics of the spacecraft. The proposals are then peer- reviewed by NASA through a set of review committees set up by NASA and consisting of astronomers and other knowledgeable members of the NASA community.
Now, NASA also has a collection of Management Operations Working Groups 'MOWGS" that advise NASA on a 1-2 times a year basis on various ongoing programmatic issues in each of the major disciplines: planetary sciences, infrared/radio, X-ray, optical astronomy etc. They usually meet for 1-2 days and consist of members of the astronomical community. Their advice can impact many aspects of mission planning and funding.
NASA can also seek advice on its overall research strategy, manned spaceflight, space station and many other programs from specially-selected panels of experts. The latest of these was the Augustine Commission which reviewed NASA's entire program and suggested new priorities consistent with their reading of the economic realities of the 1990's. Space Science was rated at the top of the list as the most productive return on investment by NASA. Dan Goldin, the current head of NASA, however, was not bound by the recommendation of the Augustine Report, and placed Space Science in the last of four priorities for NASA.
Once a set of scientific objectives and a straw man project is proposed by the astronomical community such as 'a large orbiting optical telescope in space' ( what we now call the Hubble Space Telescope), or a 'high-energy imaging gamma ray observatory'( Now called the Compton Gamma Ray Observatory), NASA has the opportunity of adding it as a new start in some future fiscal year, usually beginning with an initial Phase A study period which, if approved by congress, advances to Phase B and the actual fabrication of prototype equipment. Congress may, AT ANY TIME, discontinue funding of a scientific mission. This has happened for many programs, and has been threatened for others. Recall the Superconducting Supercollider was closed down after 25% of the installation had already been built in Texas, and billions of dollars had already been committed. NASA, too, can be forced by Congress to cancel support for important astrophysics programs. It can be resurrected ONLY by radically changing its name and objectives, hopefully without having to throw away whatever hardware had already been built.
For more information on this process close up, have a look at my For Your Information archive for updates on the progress of the 1996 budget and its impacts on scientific programs at NASA and the National Science Foundation.