Princeton/MIT SDSS Spectroscopy Home Page

(Link to imaging data.)

This page should get anyone started using the SDSS spectra within minutes. There are instructions for anyone to get the publicly-available data, or for SDSS collaborators to get the still-proprietary data. This page describes tools to view, query and use these data using IDL, SuperMongo or IRAF. There are also summary ASCII files that contain positions, magnitudes, classification information, and redshifts.

David Schlegel, schlegel@astro.princeton.edu

In this document:

Related documents:

Science web sites:

Technical web sites:

Technical mailing lists (restricted access):


Some Nifty Plots

Plot of the cumulative number of survey-quality spectra.

Here are some nifty plots of only those objects within 2 degrees of the equatorial plane:

  1. Pie diagram to z=0.15
  2. Pie diagram to z=0.6
  3. Pie diagram to z=5
  4. Logarithmically-spaced pie diagram
(On a PostScript previewer, turn off State->Antialias.) The Milky Way is masked with dashed lines. The MAIN galaxy sample are black points, the luminous red galaxies (LRGs) that are not in MAIN are red, and quasars are blue. Specifically, these plots are made with all good plates whose centers are within 0.25 deg of the plane.

Getting the Reduced Data

Which Plates Are Reduced?

The SDSS will observe approximately 2000 plates by the end of its 5 year survey. Each plate is physically an aluminum disk 60 cm in diameter that subtends 3 degrees on the sky. 640 fibers are plugged into each plate at the location of objects chosen from the photometric survey, allowing us to spectroscopically observe 640 objects at time. Typically, we integrate for 45 minutes on each plate.

Since plates are occasionally re-observed, the plate number plus the observation date uniquely identifies an observation. We track the date with modified Julian date (MJD). In these re-observations, the optical fibers may be plugged into different holes, so fiber #1 on plate 306 won't be the same on the MJD=51637 observation as on MJD=51690.

Therefore, PLATE + MJD + FIBERID uniquely determines a single observation of a single object. The TILE number uniquely determines a position on the sky, and there are a few instances where two PLATEs are drilled for one TILE.

This list of plates is auto-updated each day based upon which reductions have completed. It is available in the following formats:

As of 31 May 2002, we have observed 621 "good" plates and 22 "marginal" plates for a total of 643 * 640 = 411,520 spectra.

Aside from PLATE, TILE and MJD, this list contains the following:

RA       = Right ascension of plate center [J2000 degrees].
DEC      = Declination of plate center [J2000 degrees].
MAPNAME  = Mapping name for this plate; data with the same mapping name
           are always combined into one plate file, even if taken on
           different nights.
VERSLOG  = Version of speclog product (spectroscopic log files).
VERSUTIL = Version of idlutils product.
VERSFLAT = Version of specflat product (spectroscopic pixel flats).
VERSREAD = Version of Spectro-2D code for pre-processing raw data.
VERS2D   = Version of Spectro-2D code to reduce the individual frames.
VERSCOMB = Version of Spectro-2D code to combine frames into a plate file.
VERS1D   = Version of Princeton-1D code to measure redshifts.
VERSTARG = Version of Target code for selecting the objects on this plate.
CHUNKNAME= The name of the Target and tiling run which produced this plate.
NEXP     = Total number of frames from all cameras used in the co-adding;
           since there are 4 cameras, NEXP=12 would typically mean that there
           were 3 exposures in each camera.
NEXP_B1  = Number of exposures in b1 camera.
NEXP_B2  = Number of exposures in b2 camera.
NEXP_R1  = Number of exposures in r1 camera.
NEXP_R2  = Number of exposures in r2 camera.
EXPT_B1  = Total exposure time in b1 camera.
EXPT_B2  = Total exposure time in b2 camera.
EXPT_R1  = Total exposure time in r1 camera.
EXPT_R2  = Total exposure time in r2 camera.
SN2_G1   = (S/N)^2 per pixel at g=20.2 on spectrograph #1.
SN2_G2   = (S/N)^2 per pixel at g=20.2 on spectrograph #2.
SN2_R1   = (S/N)^2 per pixel at r=20.25 on spectrograph #1.
SN2_R2   = (S/N)^2 per pixel at r=20.25 on spectrograph #2.
SN2_I1   = (S/N)^2 per pixel at i=19.9 on spectrograph #1.
SN2_I2   = (S/N)^2 per pixel at i=19.9 on spectrograph #2.
N_GALAXY = The number of objects Princeton-1D called a galaxy.
N_QSO    = The number of objects Princeton-1D called a QSO.
N_STAR   = The number of objects Princeton-1D called a star.
N_UNKNOWN= The number of objects Princeton-1D declared spectroscopically 
           unknown (a ZWARNING flag is set)
N_SKY    = The number of sky fibers (almost always exactly 32)
N_TARGET_MAIN = Number of objects targetted for the main galaxy survey.
N_TARGET_LRG  = Number of objects targetted for the LRG galaxy survey.
N_TARGET_QSO  = Number of objects targetted for the QSO survey.
SUCCESS_MAIN = Percentage success rate for objects targetted as "main" galaxies
           and spectroscopically classified as either "GALAXY" or "QSO".
SUCCESS_LRG  = Percentage success rate for objects targetted as LRG galaxies
           and spectroscopically classified as "GALAXY".
SUCCESS_QSO  = Percentage success rate for objects targetted as QSOs
           and spectroscopically classified as "QSO".
STATUS2D = Status of Spectro-2D pipeline for reducing individual exposures;
           "Done" if done.
STATUSCOMBINE = Status of Spectro-2D pipeline for combining individual
           exposures into the spPlate files; "Done" if done.
STATUS1D = Status of Princeton-1D pipeline; "Done" if done.
QSURVEY  = Status flag set to 1 for the best observation of any tile on the sky
           that satisfies the following: (S/N)^2 > 13.0 for G1,G2,I1,I2, and
           the target version cannot be "special" or "devel".
PUBLIC   = Name of public data release; presently plates are either not public,
           or part of the Early Data Release (EDR) or Data Release 1 (DR1).

The values of SN2 give an estimate of the signal-to-noise (per pixel) of a plate at a fiducial magnitude. The fiducials are set to g=20.2, r=20.25, i=19.9. The measurement is of the signal-to-noise squared, per pixel, for an object at that magnitude. All these values must be > 13 for a plate to be "marginal". All these values must be > 15 for a plate to be "good".

Avoiding Duplicate Objects

The best effort to avoid duplicate objects is to make use of the SPECPRIMARY field in the spAll summary file. That selects the best observation of any given object.

Selecting only those plates with QSURVEY=1 will yield non-duplicate observations of tiles on the sky that meet the survey requirements for signal-to-noise. There still may be a few duplicate objects, as some objects are put on multiple tiles for "quality assurance". These duplicates can be identified with OBJTYPE="QA". That OBJTYPE column exists either in HDU #5 of the spPlate files, or in a column by that name in the spAll summary files.

Note that the above statements are almost but not exactly true. There were some target-selection errors that resulted in duplicated targets not labelled as "QA" (notably in runs 94 and 125; see sdss-spectro/710). The following IDL commands can be used to select a truly unique set of objects:

  IDL> columns = ['PLATE','FIBER','MJD','OBJTYPE','PLUG_RA','PLUG_DEC']
  IDL> spall = hogg_mrdfits('spAll-qsurvey.fits',1,columns=columns)
  IDL> spall = spall[where(strmatch(spall.objtype, 'QA*') EQ 0)]
  IDL> junk = djs_angle_group(spall.plug_ra, spall.plug_dec, 1./3600, $
       gindx=gindx, gstart=gstart, gcount=gcount)
  IDL> spall = spall[gindx[gstart]]
The above removes "QA" objects, then selects the first instance of any remaining duplicates. The procedure DJS_ANGLE_GROUP() is not speedy, and may take about an hour.

What Version of the Code Was Used?

The version of the code used in the reductions is specified in the header keywords VERS2D, VERSCOMB, VERS1D as described in the data model.

All these reductions were done using versions of the code tagged v4_9_8 through v4_10_6, which represents a fairly homogeneous set of reductions.

Copying the Princeton Reductions of the Public Data

The Princeton re-reductions of the "public release" data is available from this web site. The data are grouped into disjoint samples of when the data became available: the Early Data Release (EDR) with 139 plates (88,960 spectra), Data Release 1 (DR1) adds to this 192 plates (122,880 spectra), Data Release 2 (DR2) adds to this 279 plates (178,560 spectra), Data Release 3 (DR3) adds to this 252 plates (161,280 spectra), Data Release 4 (DR4) adds to this 479 plates (306,560 spectra), Data Release 5 (DR5) adds to this 324 plates (207,360 spectra), and Data Release 6 (DR6) adds to this 349 plates (223,360 spectra).

To download the 14 Gb of EDR data (including the spAll-EDR summary file; if you only want the spAll file, then remove "--accept *.fits" from the command):

 wget --dot-style=mega --quota=0 --no-host-directories \
   --no-parent --non-verbose --recursive --retr-symlinks \
   --accept "*.fits" --accept "spAll*" \
   http://spectro.princeton.edu/spectro_EDR/
To download the 18 Gb of data new to DR1 (including the spAll-DR1 summary file):
 wget --dot-style=mega --quota=0 --no-host-directories \
   --no-parent --non-verbose --recursive --retr-symlinks \
   --accept "*.fits" --accept "spAll*" \
   http://spectro.princeton.edu/spectro_DR1/
To download the 27 Gb of data new to DR2 (including the spAll-DR2 summary file):
 wget --dot-style=mega --quota=0 --no-host-directories \
   --no-parent --non-verbose --recursive --retr-symlinks \
   --accept "*.fits" --accept "spAll*" \
   http://spectro.princeton.edu/spectro_DR2/
To download the 24 Gb of data new to DR3 (including the spAll-DR3 summary file):
 wget --dot-style=mega --quota=0 --no-host-directories \
   --no-parent --non-verbose --recursive --retr-symlinks \
   --accept "*.fits" --accept "spAll*" \
   http://spectro.princeton.edu/spectro_DR3/
To download the 46 Gb of data new to DR4 (including the spAll-DR4 summary file):
 wget --dot-style=mega --quota=0 --no-host-directories \
   --no-parent --non-verbose --recursive --retr-symlinks \
   --accept "*.fits" --accept "spAll*" \
   http://spectro.princeton.edu/spectro_DR4/
To download the 31 Gb of data new to DR5 (including the spAll-DR5 summary file):
 wget --dot-style=mega --quota=0 --no-host-directories \
   --no-parent --non-verbose --recursive --retr-symlinks \
   --accept "*.fits" --accept "spAll*" \
   http://spectro.princeton.edu/spectro_DR5/
To download the 34 Gb of data new to DR6 (including the spAll-DR6 summary file):
 wget --dot-style=mega --quota=0 --no-host-directories \
   --no-parent --non-verbose --recursive --retr-symlinks \
   --accept "*.fits" --accept "spAll*" \
   http://spectro.princeton.edu/spectro_DR6/
For members of the SDSS collaboration only, once DR7 is available, it can be downloaded with the following command (after substituting the usual username and password):
 wget --dot-style=mega --quota=0 --no-host-directories \
   --no-parent --non-verbose --recursive --retr-symlinks \
   --accept "*.fits" --accept "spAll*" \
   http://www.astro.princeton.edu:81/spectro_DR7/ \
   --http-user=??? --http-passwd=???

Note that these reductions are not identical to the "official SDSS data release", but have several improvements and include our redshift fits, spectral classifications, velocity dispersions for galaxies, and emission line fits.

Copying the Proprietary Reductions from Princeton (alfred account)

The spectroscopic data is reduced at Princeton, typically within 12 hours of the data being taken. These reductions are available directly from Princeton as described below.

At Princeton, the top-level directory for the reduced data is:

/u/dss/spectro
Older reductions exist at:
/u/dss/spectro_v4_8

To obtain the data from Princeton you will need access to the "alfred" account. Send your SSH public key (in the file ".ssh/identity.pub") to schlegel@astro.princeton.edu. (See Mike Blanton's notes on generating ssh keys.) Once you have been added to the authorized_keys list for user "alfred", then you can copy the data with "rsync" or "scp".

Each plate has the following files associated with it:

$SPECTRO_DATA/$PLATE/spPlate-$PLATE-$MJD.fits
$SPECTRO_DATA/$PLATE/spZbest-$PLATE-$MJD.fits
$SPECTRO_DATA/$PLATE/spZline-$PLATE-$MJD.fits
$SPECTRO_DATA/$PLATE/spZall-$PLATE-$MJD.fits
where $SPECTRO_DATA refers to the top-level directory (spectro), and $PLATE refers to a subdirectory with the plate name (e.g., 0301), and $MJD refers to the date of observation (e.g., 52000). There are also some intermediate output files (e.g., spCFrame files) that can be ignored by most users.

Most people will only want the summary files, combined spectra and 1D outputs (redshifts). These total about 125 Gb for 1300 plates as of Feb 2004. The following command will copy that entire list of files (eventually). Should the copy be interrupted, the "rsync" command can be invoked again to successfully copy the remainder of the data.

   rsync -arv --rsh="ssh1" --copy-unsafe-links \
    --include "*/" \
    --include "platelist*" --include "spAll*" \ 
    --include "*spZ*.fits" --include "spPlate*.fits" \ 
    --exclude "*" \
    alfred@sdssdata.astro.princeton.edu:/u/dss/spectro/ .

How to Keep Your Data Up-to-Date

These spectroscopic reductions are completely automated with the Spectro Robot. As data are taken at Apache Point, the data are copied to Princeton, plan files are built that describe how to reduce the data, and the reductions are batched on local and remote CPU's. You can expect all spectroscopic observations to be reduced within 24 hours.

An efficient way to keep your copy of these reductions current is to use the Unix "rsync" command as described above. A cleaner implementation of this is to use the pullspectra script from the "idlspec2d" product. If you run "rsync" or "pullspectra" each day or each week, it will only copy files that have been changed or added. This is a convenient way to keep your data up-to-date without re-copying everything en-masse. Note that you need "ssh" to work without asking for a pass-phrase for "rsync" to work as a cron job (see below) -- see your system admin if you need help with that.

Finally, you could set up a "cron" job to update your copy of the data on a daily basis. This would be done by creating a cron table file (call it "cron.table" in this example). If you want to copy the typical set of spectroscopic outputs every day at 3:15 am, this file would look something like this:

15 3 * * * /my/directory/bin/pullspectra /my/data
replacing "/my/directory/bin" with your local path to the "pullspectra" command, and replacing "/my/data" with your data directory. Note that "cron" jobs do not inheret your environment variables, so it is necessary to use explicit paths. Then load this "cron" job by typing:
crontab cron.table
This example will update your copy of the data at 3:15am every morning. To stop this updating, unload the cron table with:
crontab -r

"Special" Plates

There are a number of "special" plates, designed for commissioning either the hardware or target selection for the survey. These plates are:

Summary Files

We generated one monstrous file with all the spectroscopic and photometric outputs (but not the actual images, spectra or spectroscopic line fits). These files are FITS binary tables. The photometric information is from the best-available ``uber-calibration'' of the imaging data as of April 2005. The details of the photometric information can be found here: http://photo.astro.princeton.edu/#data_model . These files include matches to to the FIRST radio catalog, the 2MASS catalog, and the USNO-B astrometric catalog. The following files exist:

  1. spAll.fits - All spectroscopic data taken to date from 'marginal' or 'good' plates (see PLATEQUALITY in the platelist file)
  2. spAll-EDR.fits - spAll generated from only those plates in the Early Data Release (EDR)
  3. spAll-DR1.fits - spAll generated from only those plates new to Data Release 1 (DR1)
  4. spAll-DR2.fits - spAll generated from only those plates new to Data Release 2 (DR2)
  5. spAll-DR3.fits - spAll generated from only those plates new to Data Release 3 (DR3)
  6. spAll-DR4.fits - spAll generated from only those plates new to Data Release 4 (DR4)
  7. spAll-public.fits - spAll generated from EDR+DR1+DR2+DR3+DR4
Although these files are quite large, most FITS readers will allow you to read such files piece-wise. There are corresponding ASCII versions of these files, i.e. spAll.dat, that contain a subset of the information.

The following information is compiled for each object:
PROGNAME Spectroscopic program: 'main' for the SDSS main survey, 'southern' for the SDSS Southern survey, and 'devel', 'special', 'photoz' for other assorted programs
CHUNKNAME Targetting chunk name (this is a numeric value for all plates in the 'main' or 'southern' survey)
PLATEQUALITY Quality of plate from platelist file: 'bad', 'marginal', or 'good'
PLATESN2 Signal-of-noise measure of the plate (the worst of the 4 cameras)
SPECPRIMARY 1 for the best spectroscopic observation of each object, 0 otherwise
SPECOBJ_ID Unique ID for spectroscopic objects, starting with 1; objects at the same coordinates have the same ID, with only one being SPECPRIMARY
NSPECOBS Number of spectroscopic observations of this object
ZANS ZANS structure from spZbest file with spectroscopic classifications (CLASS), redshifts (Z), etc.
calibObj calibObj structure with all imaging parameters; fluxes are in units of nano-maggies, so MAG=22.5-2.5*log10(FLUX)

In addition, we've also made ASCII files with the most relevant quantities that one may wish to search on. These files are "spAll*.dat", and are basically a stripped-down version of "spAll*.fits" for those FITS-challenged astronomers. I have made some small changes to the ASCII file to make it more convenient with Unix commands such as "awk" that want to index based upon word number: (1) blank strings have been replaced with double-quotes, and (2) spaces within strings have been replaced with plus signs (i.e., replacing "BROADLINE STARBURST" with "BROADLINE+STARBURST").

These photometric outputs are from the Princeton PHOTO reductions.

If you have access to the Princeton computers, these trimmed calibObj files can be found at:

  sdssdata.astro.princeton.edu:/u/dss/spectro/calibobj

Summary Files: Selecting Complete Samples

The summary files make it convenient to select complete samples. There are three complete samples that one can generate from the SDSS spectra: the "main" galaxy sample, the Luminous Red galaxy sample (LRGs), and QSOs. For complete samples, one should only use those plates that are for the PROGNAME="main" survey (not "southern", "photoz", etc.) and the primary observations of each object (SPECPRIMARY=1).

The main galaxy survey is selected from the target flags being set to GALAXY, GALAXY_BIG, or GALAXY_BRIGHT_CORE. The following bit of IDL code will select a unique list of objects targetted by the main galaxy survey that have been spectroscopically confirmed as galaxies or QSOs:

IDL> columns = ['PLATE','FIBER','MJD','OBJTYPE','PLUG_RA','PLUG_DEC', $
      'PROGNAME','SPECPRIMARY','PRIMARGET','SECTARGET','CLASS','Z','ZWARNING']
IDL> spall = hogg_mrdfits('spAll.fits',1,columns=columns)
IDL> imain = where(spall.specprimary EQ 1 $
      AND strmatch(spall.progname,'main*') $
      AND (spall.primtarget AND 2L^6 + 2L^7 + 2L^8) NE 0 $
      AND (strmatch(spall.class,'GALAXY*') OR strmatch(spall.class,'QSO*')) $
      AND spall.zwarning EQ 0)
The Luminous Red Galaxy survey is selected from the target flags being set to GALAXY_RED or GALAXY_RED_II. The following bit of IDL code will select a unique list of objects targetted by the LRG galaxy survey that have been spectroscopically confirmed as galaxies:
IDL> columns = ['PLATE','FIBER','MJD','OBJTYPE','PLUG_RA','PLUG_DEC', $
      'PROGNAME','SPECPRIMARY','PRIMARGET','SECTARGET','CLASS','Z','ZWARNING']
IDL> spall = hogg_mrdfits('spAll.fits',1,columns=columns)
IDL> ilrg = where(spall.specprimary EQ 1 $
      AND strmatch(spall.progname,'main*') $
      AND (spall.primtarget AND 2L^5 + 2L^26) NE 0 $
      AND strmatch(spall.class,'GALAXY*') $
      AND spall.zwarning EQ 0)
The QSO survey is selected from the target flags being set to QSO_HIZ, QSO_CAP, QSO_SKIRT, QSO_FIRST_CAP or QSO_FIRST_SKIRT. (The flags QSO_MAG_OUTLIER and QSO_REJECT do not define a complete sample). The following bit of IDL code will select a unique list of objects targetted by the QSO survey that have been spectroscopically confirmed as QSOs:
IDL> columns = ['PLATE','FIBER','MJD','OBJTYPE','PLUG_RA','PLUG_DEC', $
      'PROGNAME','SPECPRIMARY','PRIMARGET','SECTARGET','CLASS','Z','ZWARNING']
IDL> spall = hogg_mrdfits('spAll.fits',1,columns=columns)
IDL> iqso = where(spall.specprimary EQ 1 $
      AND strmatch(spall.progname,'main*') $
      AND (spall.primtarget AND 2L^0 + 2L^1 + 2L^2 +2L^3 + 2L^4) NE 0 $
      AND strmatch(spall.class,'QSO*') $
      AND spall.zwarning EQ 0)

These same selections can be made from the ASCII version of the spAll summary files using the Unix "awk" command. To select the main sample of galaxies (the following must all be on one line):

% awk '{if (($22==1) && ($21=="main") && (and($19,2^6+2^7+2^8) != 0)
   && ($4=="GALAXY" || $4=="QSO") && ($8==0)) {print $0}}' spAll.dat > galaxies.dat
To select the LRG sample of galaxies (the following must all be on one line):
% awk '{if (($22==1) && ($21=="main") && (and($19,2^5+2^26) != 0)
   && ($4=="GALAXY") && ($8==0)) {print $0}}' spAll.dat > lrgs.dat
To select the QSO sample (the following must all be on one line):
% awk '{if (($22==1) && ($21=="main") && (and($19,2^0+2^1+2^2+2^3+2^4) != 0)
   && ($4=="QSO") && ($8==0)) {print $0}}' spAll.dat > qsos.dat

As of 31 May 2002, the above would select complete samples of 185,899 galaxies, 51,417 LRGs, and 22,245 QSOs.

Looking at the Data

Since standard FITS files are used, there are many ways to access these data. Below are descriptions of specially-written IDL, SM, and IRAF code for looking at the data. If you write any such tools, it would be useful to make them available to the collaboration by posting them on this site.

The absolute simplest way to look at the data is to use ATV, SAOimage or XIMTOOL to display the plate images (spPlate*.fits), which will show you all 640 spectra on a plate at once.

Setting Up IDL Tools

You must have "idl" in your path, and the IDL_PATH environment variable should already be set. At Princeton, you would type the following for C-shell:

  source /usr/peyton/common/licensed/idl/bin/idl_setup
  setenv IDL_PATH         .:+{$IDL_DIR}/lib
or the following for Bourne shell:
  IDL_DIR=/usr/peyton/common/licensed/idl
  IDL_PATH=.:+{$IDL_DIR}/lib
  alias idl=$IDL_DIR/bin/idl

If you have someone maintaining survey code at your institution (i.e., at Princeton using a Linux box), you can set up the tools by simply typing the following for C-shell:

  setenv EUPS_FLAVOR Linux
  source /u/dss/products/evilups/bin/setups.csh  # If you are using C-shell
  setup idlspec2d
or the following for Bourne-shell:
  export SHELL=/bin/bash
  export EUPS_FLAVOR=Linux
  source /u/dss/products/evilups/bin/setups.sh  # If you are using Bourne-shell
  setup idlspec2d

If you need to install the code yourself, you need two products: "idlutils", and "idlspec2d". If you wish to reduce the raw data from scratch, you will also need a third product called "specflat". See the instructions for installing this code at:

  http://spectro.princeton.edu/idlspec2d_install.html
Included are the instructions for installing from a tar file. The tar files are available at: http://spectro.princeton.edu/tarballs. The code will work on any platform (Linux, SGI, Solaris, Mac OS, etc) as long as you also have an IDL license.

Extensive documentation exists for the commands described below, in the file:

$IDLSPEC2D_DIR/doc/www/idlspec2d_doc.html

Displaying the Spectra From IDL

First, set up the "idlutils" and "idlspec2d" software, as described above.

You need to set an environment variable that points to the top-level directory of the data. At Princeton, this would be:

setenv SPECTRO_DATA /u/dss/spectro

Start IDL with:

idl

Then from within IDL, you can display individual spectra as following. To display plate 401, fiber 100:

IDL> plotspec, 401, 100
The spectrum is shown in white, the errors in red (except masked points are set to zero), and the best-fit eigenspectrum in blue. The mouse buttons will zoom in (left), recenter (center), or zoom out (right). The frame can be saved as a PostScript file by selecting File->WriteEPS from the left-hand corner.

Make the same plot, but boxcar-smooth the spectrum:

IDL> plotspec, 401, 100, nsmooth=10

Some plates are observed on multiple nights. To select one of the two observations of plate 306:

IDL> plotspec, 306, 20, mjd=51690

You can write the spectrum to an ASCII file as follows:

IDL> writespec, 401, 100
The default file name will be "spec-0401-51788-100.dat". This can be changed by setting the FILENAME keyword:
IDL> writespec, 401, 100, filename='junk.dat'

Quitting IDL:

IDL> exit

For IDL pundits, the general-purpose routine for reading the the coadded spectra is READSPEC, or SPFRAME_READ for reading the spectra from a single exposure.

Displaying the Spectra From SM

The SM tools were written by Michael Strauss, Vijay Narayanan, and Robert Lupton.

Problems and suggestions should be reported to Michael Strauss (strauss@astro.princeton.edu).

You will need a version of SM which supports FITS extensions (version SM2_4_8 or later). If you have a valid SM license, you or your system admin should be able to install this (see the bottom of Michael's README file.

At Princeton, you could type the following to display plate 306 fiber 100, then overplot the best-fit template:

setenv SPECTRO_DATA /u/dss/spectro
setenv IDLSPEC2D_DIR /u/schlegel/idlspec2d
cd ~strauss/spectro_sm/release
sm
dev x11
input_new spectro.sm
define idl 1
plotspec 306 100
   t

Displaying the Spectra From IRAF

Described below is the most basic method of plotting the spectra from IRAF. Better tools will be posted on this page as they are available.

Use the splot task in the noao.onedspec package. For example:

cl> noao
no> onedspec
on> splot spPlate-0311-51665.fits[0][*,1]   # plot fiber 1's flux
on> splot spPlate-0311-51665.fits[1][*,1]   # plot fiber 1's inverse variance **
on> splot spPlate-0311-51665.fits[0]        # interactively plot fluxes:
Image line/aperture to plot (0:) (1): 65    # start w/fiber 65; use # to choose
        # a new fiber, or ( and ) to move down and up by 1 in number; q to quit
on> phelp splot                             # for detailed help on splot
on> phelp onedspec                          # for summary of other tasks, e.g.:
on> specplot spPlate-0311-51665.fits[0][*,1],spPlate-0311-51665.fits[0][*,2]
        # stack and plot fibers 1 and 2 without overlapping

Pat Hall's IRAF SDSS Tasks

Pat Hall has written an IRAF package, PHIST, for displaying SDSS spectra. It does not have all the functionality of the IDL tools, but does have some complementary functions. The task list is as follows:

 dec2seg - convert decimal coordinates to sexigesimal
   doigi - run existing IGI script
hexprint - convert a decimal number to hexadecimal
  ionlte - calculate LTE ratios of MgI-III,AlI-IV,FeI-IV for given n_e,T ranges
  lplots - overplot wavelengths of emission / absorption lines for given z
 lplots2 - 2-panel lplots
mkfitsobs - make coadded FITS and text spectrum
   mkobs - make text spectrum from FITS spectrum
   sbqgr - sample task for converting GR's SX .tbl output to formatted ASCII
   sbqsx - sample task for converting SX .tbl output to formatted ASCII
  sbqsx2 - another task for converting SX .tbl output to formatted ASCII
 seg2dec - convert sexigesimal coordinates to decimal
  spdata - get objid,mags,ra,dec,objtype from spectrum (plate,mjd,fiber)
   splat - interactively SPLOT a specific observation (plate+mjd+fiber) 
   splet - non-interactively SPECPLOT an observation and its error spectrum 
unredden - deredden spectrum using SMC or LMC extinction curve
   vcalc - calculate velocity offset in quasar rest frame
   vplot - overplot wavelengths of emission / absorption lines in velocity space
  vplots - multiple vplot
   wcalc - calculate vacuum wavelength from air wavelength
   zplot - plot spectrum in the rest frame at the specified z
Here's the README file on how to retrieve & install the package for version v.2 (December 2001):
        PHIST -- Pat Hall's IRAF SDSS Tasks

v.2     011207  Read "phist.hlp" ("phelp phist opt=sys" in IRAF) for an
        introduction.  Entire package available as a compressed tarfile
        from http://astro.princeton.edu/~pathall/phist2.tar.gz
        To install, enter the proper pathnames in "phist.cl" and
        "phist.hd", load the IRAF "softools" package, and type
        "mkhelpdb root.hd helpdb.mip", deleting the old helpdb.mip
        if necessary.  See "phist.hlp" for how to include
        the package in your default IRAF startup.

What Is the Data Model?

Intermediate outputs: Final, co-added spectra: Spectroscopic classifications, redshifts, and line fitting:

spCFrame-cc-eeeeeeee.fits

This file contains the extracted spectra for a single CCD of a single exposure. In general, each object has several blue exposures and several red exposures. These spectra are still in the native wavelength mapping, which is neither linear in wavelength nor log-wavelength. However, the spectra are flux-calibrated, and deviant pixels are rejected in the pixel mask (from the comparison with the other exposures).

The spCFrame file contains the following:

HDU #0 = Flux in units of 10^(-17) erg/s/cm^2/Ang [FLOAT]
HDU #1 = Inverse variance (1/sigma^2) for the above [FLOAT]
HDU #2 = Pixel mask [32-bit INT]
HDU #3 = Wavelength solution in log-10 Angstroms (vacuum) [FLOAT]
HDU #4 = Wavelength dispersion in units of (10^(-4) log-10 wavelength)^(-1) [FLOAT]
         (these are the same units as in the combined-spectrum spPlate file)
HDU #5 = Plug-map structure from plPlugMapM file [BINARY FITS TABLE]
HDU #6 = Sky flux in units of 10^(-17) erg/s/cm^2/Ang [FLOAT]
HDU #7 = X position on the CCD [FLOAT]

spDiag2d-pppp-mmmmm.log

The log file from running SPREDUCE2D for one night of data for one plate.

spDiag2d-pppp-mmmmm.ps

The PostScript plots associated with the spDiag2d-pppp-mmmmm.log file.

spPlate-pppp-mmmmm.fits

This file contains the combined spectra for all exposures (potentially spanning multiple nights) for a given mapped plate. The first 5 HDU's contain NPIX by 640 images, where NPIX is the number of wavelengths. The pixel scale is exactly 1e-4 in log-wavelength (any units), which corresponds to 1e-4 * c * ln(10) = 69.029765 km/sec.

The spPlate file contains the following:

HDU #0 = Flux in units of 10^(-17) erg/s/cm^2/Ang [FLOAT]
HDU #1 = Inverse variance (1/sigma^2) for the above [FLOAT]
HDU #2 = AND mask [32-bit INT]
HDU #3 = OR mask [32-fit INT]
HDU #4 = Wavelength dispersion in pixels [FLOAT]
HDU #5 = Plug-map structure from plPlugMapM file [BINARY FITS TABLE]
HDU #6 = Average sky flux in units of 10^(-17) erg/s/cm^2/Ang [FLOAT]

The following header keywords are of interest (in HDU #0):

PLATEID = Plate ID
TILEID  = Tile ID
MJD     = Modified Julian date for first night of observation
MJDLIST = Modified Julian date for all nights of observation
MAPID   = Fiber mapper ID
CARTID  = Fiber cartridge used at the telescope
NAME    = Name of target: Plate ID, MJD, Fiber mapper rerun
COEFF0  = Central wavelength (log10-Angstroms) of first pixel
COEFF1  = Wavelength range of each pixel (log-10 Angstroms)
CRVAL1  = (Same as COEFF0)
CD1_1   = (Same as COEFF1)
RADEG   = Right ascension of plate center (J2000 degrees)
DECDEG  = Declination of plate center (J2000 degrees)
AIRMASS = Airmass for mid-point of observation
RA      = Right ascension of where the telescope thinks it is pointing
DEC     = Declination of where the telescope thinks it is pointing
AZ      = Azimuth of the telescope (degrees)
ALT     = Altitude of the telescope (degrees)
NEXP    = Total number of frames from all cameras used in the co-adding;
          since there are 4 cameras, NEXP=12 would typically mean that there
          were 3 exposures in each camera.
EXPT_B1 = b1 camera exposure time (seconds)
EXPT_B2 = b2 camera exposure time (seconds)
EXPT_R1 = r1 camera exposure time (seconds)
EXPT_R2 = r2 camera exposure time (seconds)
EXPTIME = Minimum of exposure times for all cameras
SPEC1_G = (S/N)^2 per pixel at g=20.2 on spectrograph #1
SPEC2_G = (S/N)^2 per pixel at g=20.2 on spectrograph #2
SPEC1_R = (S/N)^2 per pixel at r=20.25 on spectrograph #1
SPEC2_R = (S/N)^2 per pixel at r=20.25 on spectrograph #2
SPEC1_I = (S/N)^2 per pixel at i=19.9 on spectrograph #1
SPEC2_I = (S/N)^2 per pixel at i=19.9 on spectrograph #2
GOFFSTD = Median spectrophotometry minus photo mag for standard stars in g-band
ROFFSTD = Median spectrophotometry minus photo mag for standard stars in r-band
IOFFSTD = Median spectrophotometry minus photo mag for standard stars in i-band
GOFFRMS = RMS of spectrophotometry minus photo mag for standard stars in g-band
ROFFRMS = RMS of spectrophotometry minus photo mag for standard stars in r-band
IOFFRMS = RMS of spectrophotometry minus photo mag for standard stars in i-band
GOFFGAL = Median spectrophotometry minus photo mag for "main" galaxies in g-band
ROFFGAL = Median spectrophotometry minus photo mag for "main" galaxies in r-band
IOFFGAL = Median spectrophotometry minus photo mag for "main" galaxies in i-band
GOFFGAL = RMS of spectrophotometry minus photo mag for "main" galaxies in g-band
ROFFGAL = RMS of spectrophotometry minus photo mag for "main" galaxies in r-band
IOFFGAL = RMS of spectrophotometry minus photo mag for "main" galaxies in i-band
BESTEXP = Exposure number for the "best" exposure (all 4 cameras)
EXPIDxx = Identifier for the each invididual exposure+CCD used in this combine,
          for ex. 'b1-00006853-00006857-00006858' refers to the b1 CCD,
          exposure #6853, using flat #6857 and arc #6858
TAI     = Number of seconds since Nov 17 1858 for mid-point of observation
TAI-BEG = Number of seconds since Nov 17 1858 for beginning of observation
TAI-END = Number of seconds since Nov 17 1858 for end of observation
SEEING20= 20th-percentile of seeing during exposure (arcsec)
SEEING50= 50th-percentile of seeing during exposure (arcsec)
SEEING80= 80th-percentile of seeing during exposure (arcsec)
RMSOFF20= 20th-percentile of RMS offset of guide fibers (arcsec)
RMSOFF50= 50th-percentile of RMS offset of guide fibers (arcsec)
RMSOFF80= 80th-percentile of RMS offset of guide fibers (arcsec)
VERSIDL = Version of IDL
VERS2D  = Version of idlspec2d for 2D reduction
VERSCOMB= Version of idlspec2d for combining multiple spectra
VERSFLAT= Version of specflat for spectroscopic flat-fields
VERS1D  = Version of idlspec2d for 1D reduction
HELIO_RV= Barycentric velocity correction (km/sec)

There are two masks, an "AND" mask and an "OR" mask. The spectra are constructed from 3 or more 15-minute observations, and the "AND" mask bits are set if that bit is set for each and every input observation. The "OR" mask bits are set if that bit is set for any of the observations. Usually, I only look at the "AND" mask.

The mask bits are set as follows:

 0 NOPLUG          Fiber not listed in plugmap file
 1 BADTRACE        Bad trace from routine TRACE320CRUDE
 2 BADFLAT         Low counts in fiberflat
 3 BADARC          Bad arc solution
 4 MANYBADCOLUMNS  >10% pixels are bad columns
 5 MANYREJECTED    >10% pixels are rejected in extraction
 6 LARGESHIFT      Large spatial shift between flat and object pos'n
 7 BADSKYFIBER     Sky Fiber shows extreme residuals
 8 NEARWHOPPER     Within 2 fibers of a whopping fiber (inclusive)
10 SMEARIMAGE      Smear available for red and blue cameras (deprecated)
11 SMEARHIGHSN     S/N sufficient for full smear fit (deprecated)
12 SMEARMEDSN      S/N only sufficient for scaled median fit (deprecated)
16 NEARBADPIXEL    Bad pixel within 3 pixels of trace
17 LOWFLAT         Flat field less than 0.5
18 FULLREJECT      Pixel fully rejected in extraction (INVVAR=0)
19 PARTIALREJECT   Some pixels rejected in extraction
20 SCATTEREDLIGHT  Scattered light significant
21 CROSSTALK       Cross-talk significant
22 NOSKY           Sky level unknown at this wavelength (INVVAR=0)
23 BRIGHTSKY       Sky level > flux + 10*(flux error)
                   AND sky > 2.0 * median(sky,99 pixels)
24 NODATA          No data available in combine B-spline (INVVAR=0)
25 COMBINEREJ      Rejected in combine B-spline
26 BADFLUXFACTOR   Low flux-calibration or flux-correction factor
27 BADSKYCHI       Relative chi^2 > 3 in sky residuals at this wavelength
28 REDMONSTER      Contiguous region of bad chi^2 in sky residuals
                   (with threshhold of relative chi^2 > 3)

When low numbered bits (<16) are set, those will be set for half of the spectra -- either the blue or red spectrograph. The higher-numbered bits (>=16) are set for individual pixels.

Which mask bits are important? The conditions that are considered very bad are already used to set the errors to infinity for the effected pixels (specifically, the inverse variance is set to zero). The most useful mask bit to look at is BRIGHTSKY, which indicates when the sky is so bright relative to the object that perhaps one shouldn't trust any of the object flux there. Our reported errors are meant to include sky-subtraction errors, but there are instances (particularly around 5577) where these errors may be untrustworthy.

Dispersion and sky: The dispersion per pixel and the sky flux are computed at each pixel by re-weighting the individual spectra at each pixel according to their formal errors. This re-weighting is only approximate.

Sky wavelengths: Note that the sky lines are slightly shifted in the reductions because we transform the velocities to the barycenter of the solar system. Each exposure that contributes to the co-added spectra will have slightly different barycenter correction, so the "average sky" contains a superposition of these slightly-offset sky lines. These shifts keep the object spectra as-measured at the barycenter, regardless of the time of year or the Earth's rotation relative to the spectroscopic targets.

spDiagcomb-pppp-mmmmm.log

The log file from running SPCOMBINE for one plate (possibly combining data from multiple nights if the fibers have the same plugging).

spDiagcomb-pppp-mmmmm.ps

The PostScript plots associated with the spDiagcomb-pppp-mmmmm.log file.

spZbest-pppp-mmmmm.fits

This file contains the spectroscopic classifications and redshifts from the Princeton-1D code. Ths first HDU is the FITS binary table described below. The 2nd HDU is the best-fit eigen-spectrum for each object. These can be regenerated from the tabular information, but for convenience I've explicitly computed them.

The columns in the first HDU are as follows:

PLATE      Plate number
TILE       Tile number
MJD        Modified Julian date of observation
FIBERID    Fiber ID (1 to 640)
OBJID      4-element PHOTO ID of targetted object: run, re-run, column, ID
OBJTYPE    Why this object was targetted.  Note that if this field says QSO,
           it could be the case that this object would have been targetted
           as a GALAXY or any number of other categories as well.  The
           PRIMTARGET and SECTARGET flags in the plug-map structure (in the
           spPlate file) gives this full information.
PLUG_RA    Object RA (drilled fiber position) [J2000 degrees].
PLUG_DEC   Object DEC (drilled fiber position) [J2000 degrees].
CLASS      Spectro classification: GALAXY, QSO, STAR; reductions prior to
           version v4_7 will also use SKY and UNKNOWN; later versions set
           the ZWARNING flags
SUBCLASS   Spectro sub-classification.  Galaxies and QSO's can be given
             sub-classifications based upon their emission lines.
             This text includes the word "AGN" if:
               log10(OIII/Hbeta) > 0.7 - 1.2 * (log10(NII/Halpha) - 0.4)
             This text includes the word "STARFORMING" if:
               log10(OIII/Hbeta) < 0.7 - 1.2 * (log10(NII/Halpha) - 0.4)
             If the H_alpha E.W. > 500 Ang, then upgrade "STARFORMING" to
             "STARBURST."
             In all cases for AGN, STARFORMING, and STARBURST, the 4 emission
             lines used for that sub-classification must have well-measured
             errors (e.g., LINEAREA_ERR > 0).
             If any galaxies or quasars have lines detected at the 10-sigma
             level with sigmas > 200 km/sec at the 5-sigma level, call them
             "BROADLINE".
           For stars, the following subclassifications are in use:
             O, OB, B6, B9, A0, A0p, F2, F5, F9, G0, G2, G5, K1, K3, K5, K7,
             M0V, M2V,M1, M2, M3, M4, M5, M6, M7, M8, L0, L1, L2, L3, L4, L5,
             L5.5, L9, T2, Carbon, Carbon_lines, CarbonWD, CV.
Z          Redshift; if the ZWARNING flag is nonzero, then assume that this
           redshift is incorrect
Z_ERR      Redshift error based upon fit to chi^2 minimum; negative for
           invalid fit
RCHI2      Reduced chi^2 for best fit
DOF        Degrees of freedom for best fit
RCHI2DIFF  Difference in reduced chi^2 of best solution to 2nd best solution
TFILE      Template file in $IDLSPEC2D_DIR/templates
TCOLUMN    Columns to use in template file (0-indexed); unused values set to -1
NPOLY      Number of polynomial terms used in conjunction with TFILE
THETA      Eigenvalue coefficients for each column in template file + each polynomial term
THETA_COVAR  Covariance matrix for THETA, where the square root of the
           diagonal elements gives the formal 1-sigma statistical errors
VDISP      Velocity dispersion, only computed for galaxies (km/sec)
           and explicitly masking emission line regions
VDISP_ERR  Error in VDISP (km/sec); negative for invalid fit
VDISPZ     Redshift for best-fit velocity dispersion
VDISP_ERR  Error in VDISPZ
VDISPCHI2  Chi^2 for best-fit velocity dispersion
VDISPNPIX  Number of pixels overlapping the templates used in the
           velocity dispersion fit
VDISPDOF   Degrees of freedom for best-fit velocity dispersion, equal to
           VDISPNPIX minus the number of templates minus the number of
           polynomial terms minus 1 (the last 1 is for the velocity dispersion)
WAVEMIN    Minimum observed (vacuum) wavelength for this object (Angstroms)
WAVEMAX    Maximum observed (vacuum) wavelength for this object (Angstroms)
WCOVERAGE  Amount of wavelength coverage in log-10(Angstroms)
ZWARNING   A flag set for bad redshift fits in place of calling CLASS=UNKNOWN.
           The following bits may be set:
           0 = Sky fiber
           1 = Too little wavelength coverage (WCOVERAGE < 0.18)
           2 = Chi^2 is too close to chi^2 for the next best fit (within 0.01
               in the reduced chi^2)
           3 = Synthetic spectrum is negative (only set for stars and QSO's)
           4 = Fraction of points above 5 sigma is too large (> 5%)
           5 = Chi^2 minimum was at the edge of the redshift-fitting range,
               resulting in Z_ERR being set to -1
           6 = Negative emission in a QSO line, triggered only in QSO spectra
               iff C_IV, C_III, Mg_II, H_beta, or H_alpha has
               LINEAREA + 3 * LINEAREA_ERR < 0
SN_MEDIAN  Median S/N for all good pixels
CHI68P     68-th percentile value of abs(chi) of the best-fit synthetic
           spectrum to the actual spectrum, which would be 1.0 for a good fit
           if the errors are properly normalized.
FRACNSIGMA Fraction of pixels deviant by more than N sigma, where N=[1,2,...10]
           (igorning all points blueward of rest-frame 1216 Ang)
FRACNSIGHI Fraction of pixels high by more than N sigma, where N=[1,2,...10]
           (igorning all points blueward of rest-frame 1216 Ang)
FRACNSIGLO Fraction of pixels low by more than N sigma, where N=[1,2,...10]
           (igorning all points blueward of rest-frame 1216 Ang)
           (incorrectly computed for versions of the code before v4_9_10)
SPECTROFLUX 5-element array of integrated flux in each of the 5 SDSS imaging
            filters (ugriz); the units are nanomaggies, which is 1 at 22.5
            magnitude; convert to magnitudes with
              22.5 - 2.5 * LOG_10(SPECTROFLUX);
            the u-band and z-band counts should not be used
SPECTROFLUX_IVAR: The approximate error (inverse variance) in SPECTROFLUX
            as measured by the sky fibers; we assign identical errors
            in linear flux units to all fibers on a plate.
SPECTROSYNFLUX: Same as SPECTROFLUX, but measured using the best-fit
                synthetic eigen-spectrum rather than the actual data points
SPECTROSYNFLUX_IVAR: The approximate error (inverse variance) in SPECTROSYNFLUX
            as measured by the sky fibers; we assign identical errors
            in linear flux units to all fibers on a plate.
SPECTROSKYFLUX: 5-element array of the sky flux in each of the 5 SDSS imaging
                filters
ANYANDMASK Mask bits which are set if any pixels for an object's ANDMASK
           have that bit set (these fields incorrect in current reductions)
ANYORMASK  Mask bits which are set if any pixels for an object's ORMASK
           have that bit set (these fields incorrect in current reductions)
SPEC1_G    (S/N)^2 at g=20.20 for spectrograph #1 (same value for 320 fibers)
SPEC1_R    (S/N)^2 at r=20.25 for spectrograph #1 (same value for 320 fibers)
SPEC1_I    (S/N)^2 at i=19.90 for spectrograph #1 (same value for 320 fibers)
SPEC2_G    (S/N)^2 at g=20.20 for spectrograph #2 (same value for 320 fibers)
SPEC2_R    (S/N)^2 at r=20.25 for spectrograph #2 (same value for 320 fibers)
SPEC2_I    (S/N)^2 at i=19.90 for spectrograph #2 (same value for 320 fibers)
ELODIE_FILENAME   File name for best-fit Elodie star
ELODIE_OBJECT     Star name (mostly Henry Draper names)
ELODIE_SPTYPE     Spectral type
ELODIE_BV         (B-V) color
ELODIE_TEFF       Effective temperature
ELODIE_LOGG       Log10(gravity)
ELODIE_FEH        [Fe/H]
ELODIE_Z          Redshift
ELODIE_Z_ERR      Redshift error; negative for invalid fit
ELODIE_Z_MODELERR The standard deviation in redshift amongst the 12
                  best-fit stars
ELODIE_RCHI2      Reduced chi^2
ELODIE_DOF        Degrees of freedom for fit
VELOCITY_TAI      Timestamp for each exposure [sec]
VELOCITY          Velocity for each exposure [km/s], where the first exposure
                  should be zero within roundoff
VELOCITY_ERR      Error in VELOCITY [km/s]
VELOCITY_COVAR    Covariance matrix for VELOCITY, where the square root of the
                  diagonal elements gives the formal 1-sigma statistical errors

spZline-pppp-mmmmm.fits

This file contains the line fits from from the Princeton-1D code.

The columns in the first HDU are as follows:

PLATE      Plate number
MJD        Modified Julian date of observation
FIBERID    Fiber ID (1 to 640)
LINENAME   Line name
LINEWAVE   Catalog wavelength for this line in vacuum Angstroms
LINEZ          Redshift
LINEZ_ERR      Redshift error (negative for invalid fit)
LINESIGMA      Gaussian width in km/sec
LINESIGMA_ERR  Error in gaussian width (negative for invalid fit)
LINEAREA       Area in gaussian fit where units are (flux-units) * Ang
LINEAREA_ERR   Flux error (negative for invalid fit)
LINEEW         Equivalent width (Angstroms)
LINEEW_ERR     Equivalent width error (negative for invalid fit)
LINECONTLEVEL      Continuum level at line center
LINECONTLEVEL_ERR  Error in continuum level at line center
LINENPIX           Number of good pixels within +/- 3 sigma of the line center
LINEDOF            Degrees of freedom in fit, approximated as LINENPIX minus
                   the number of terms fit for that line, which can be
                   fractional if one parameter if fixed betwen several lines
LINECHI2           Chi^2 for all points within +/- 3 sigma of the line center
                   (negative if no such points)

The following lines are fit. Lines with the same ZINDEX are constrained to have the same redshift. Lines with the same WINDEX are constrained to have the same gaussian width. The flux ratios of [O_III] and [N_II] are are constrained to be 1:3.
LINEWAVE LINENAME ZINDEX WINDEX Flux ratio
1215.67 Ly_alpha z_lya w_ly_a
1240.81 N_V 1240 zemission w_n_v
1549.48 C_IV 1549 zemission wemission
1640.42 He_II 1640 zemission wemission
1908.734 C_III] 1908 zemission wemission
2799.49 Mg_II 2799 zemission wemission
3726.032 [O_II] 3725 zemission wemission
3728.815 [O_II] 3727 zemission wemission
3868.75 [Ne_III] 3868 zemission wemission
3889.049 H_epsilon zemission w_balmer
3970.00 [Ne_III] 3970 zemission wemission
4101.734 H_delta zemission w_balmer
4340.464 H_gamma zemission wemission
4363.209 [O_III] 4363 zemission wemission
4685.68 He_II 4685 zemission wemission
4861.325 H_beta zemission w_balmer
4958.911 [O_III] 4959 zemission wemission
5006.843 [O_III] 5007 zemission wemission 3 * ([O_III] 4959)
5411.52 He_II 5411 zemission wemission
5577.339 [O_I] 5577 zemission wemission
6300.304 [O_I] 6300 zemission wemission
6363.776 [O_I] 6363 zemission wemission
6548.05 [N_II] 6548 zemission wemission
6562.801 H_alpha zemission w_balmer
6583.45 [N_II] 6583 zemission wemission 3 * ([N_II] 6548)
6716.44 [S_II] 6716 zemission wemission
6730.82 [S_II] 6730 zemission wemission
7135.790 [Ar_III] 7135 zemission wemission

spZall-pppp-mmmmm.fits

This file is a FITS binary table identical in form to the spZbest file, but with all the fits ever done to each object. This includes 5 possible galaxy redshifts, 5 possible QSO redshifts, and 1 for each of many types of stars. They are rank-sorted for each object, letting you go through and keep asking "but what's the next best fit?". This is what the IDL procedure plotspec uses when the ZNUM keyword is specified. For example, ZNUM=2 will over-plot the 2nd-best fit eigenspectrum.

For the v4_9_8 through v5_0_0 reductions, NAXIS2=33280 in HDU#1 for the spZall files, meaning there are fits for 33280/640 = 52 sets of eigentemplates per object. Most of those are single-eigenspectra stars.

Not all of the parameters in the spZbest file are computed for the spZall file. For example, the SDSS filter convolutions are not done.

spDiag1d-pppp-mmmmm.log

The log file from running the Princeton-1D reduction code (SPREDUCE1D) on a plate.

spDiag1d-pppp-mmmmm.ps

The PostScript plots associated with the spDiag1d-pppp-mmmmm.log file.

What Are the Target Flags?

The target selection sets bits in two flags called PRIMTARGET and SECTARGET. An object can have multiple bits set. For example, if the target selection code determined that an object was both a QSO_SKIRT and a STAR_WHITE_DWARF, then two bits will be set. The value of PRIMTARGET in this example will be 2^4 + 2^19 = 524304.

Let's say you want to find all objects either targetted as a carbon star, or spectroscopically classified as one. All this information is in the summary file "spAll.fits". From IDL, one could search for these objects and print their PLATE, MJD, and FIBERID as follows:

IDL> a = mrdfits('spAll.fits',1) ; This file should be in your current directory
IDL> i = where((a.primtarget AND 2L^14) NE 0)
IDL> print, transpose([[a[i].plate],[a[i].mjd],[a[i].fiberid]])

     [ Prints list of objects targetted as a Carbon star.]

IDL> i = where(strmatch(a.subclass, 'Carbon*') AND a.zwarning EQ 0)
IDL> print, transpose([[a[i].plate],[a[i].mjd],[a[i].fiberid]])

     [ Prints list of objects spectroscopically called a Carbon star.]

Now plot all of the objects found above:

IDL> i=0 & plotspec, a.plate, a.fiberid, mjd=a.mjd

The PRIMTARGET bit values are as follows (in hex and decimal):

  QSO_HIZ           = 0x1          = bit # 0 =        1
  QSO_CAP           = 0x2          = bit # 1 =        2
  QSO_SKIRT         = 0x4          = bit # 2 =        4
  QSO_FIRST_CAP     = 0x8          = bit # 3 =        8
  QSO_FIRST_SKIRT   = 0x10         = bit # 4 =       16
  QSO_MAG_OUTLIER   = 0x2000000    = bit #25 = 33554432
  QSO_REJECT        = 0x20000000   = bit #29 =536870912
  GALAXY_RED        = 0x20         = bit # 5 =       32
  GALAXY_RED_II     = 0x4000000    = bit #26 = 67108864
  GALAXY            = 0x40         = bit # 6 =       64
  GALAXY_BIG        = 0x80         = bit # 7 =      128
  GALAXY_BRIGHT_CORE= 0x100        = bit # 8 =      256
  ROSAT_A           = 0x200        = bit # 9 =      512
  ROSAT_B           = 0x400        = bit #10 =     1024
  ROSAT_C           = 0x800        = bit #11 =     2048
  ROSAT_D           = 0x1000       = bit #12 =     4096
  ROSAT_E           = 0x8000000    = bit #27 =134217728
  STAR_BHB          = 0x2000       = bit #13 =     8192
  STAR_CARBON       = 0x4000       = bit #14 =    16384
  STAR_BROWN_DWARF  = 0x8000       = bit #15 =    32768
  STAR_SUB_DWARF    = 0x10000      = bit #16 =    65536
  STAR_CATY_VAR     = 0x20000      = bit #17 =   131072
  STAR_RED_DWARF    = 0x40000      = bit #18 =   262144
  STAR_WHITE_DWARF  = 0x80000      = bit #19 =   524288
  STAR_PN           = 0x10000000   = bit #28 =268435456
  SERENDIP_BLUE     = 0x100000     = bit #20 =  1048576
  SERENDIP_FIRST    = 0x200000     = bit #21 =  2097152
  SERENDIP_RED      = 0x400000     = bit #22 =  4194304
  SERENDIP_DISTANT  = 0x800000     = bit #23 =  8388608
  SERENDIP_MANUAL   = 0x100000     = bit #24 = 16777216
  QSO_MAG_OUTLIER   = 0x200000     = bit #25 = 33554432
  GALAXY_RED_II     = 0x400000     = bit #26 = 67108864
  ROSAT_E           = 0x800000     = bit #27 =134217728
  STAR_PN           = 0x1000000    = bit #28 =268435456
  QSO_REJECT        = 0x2000000    = bit #29 =536870912
  SOUTHERN_SURVEY   = 0x8000000    = bit #31 =-2147483648

The SECTARGET bit values are as follows (in hex and decimal):

  LIGHT_TRAP        = 0x1          = bit # 0 =        1
  REDDEN_STD        = 0x2          = bit # 1 =        2
  TEST_TARGET       = 0x4          = bit # 2 =        4
  QA                = 0x8          = bit # 3 =        8
  SKY               = 0x10         = bit # 4 =       16
  SPECTROPHOTO_STD  = 0x20         = bit # 5 =       32
  GUIDE_STAR        = 0x40         = bit # 6 =       64
  BUNDLE_HOLE       = 0x80         = bit # 7 =      128
  QUALITY_HOLE      = 0x100        = bit # 8 =      256
  HOT_STD           = 0x200        = bit # 9 =      512
  SOUTHERN_SURVEY   = 0x8000000    = bit #31 =-2147483648

These flag values were copied from the "target" module files "common/arTargetType.h" and "etc/diags.tcl". (There doesn't actually appear to be any single file that tracks all of these flags!)

The Wavelengths Are in Vacuum!!??

Yes, the data are stored using vacuum wavelengths. This makes sense for quasologists. But most optical astronomers know the wavelengths of transitions as measured at S.T.P., which is how the CRC lists them for any transitions redward of 2000 Angstroms.

The IAU standard for conversion from air to vacuum wavelengths is given in Morton (1991, ApJS, 77, 119). For vacuum wavelengths (VAC) in Angstroms, convert to air wavelength (AIR) via:

  AIR = VAC / (1.0 + 2.735182E-4 + 131.4182 / VAC^2 + 2.76249E8 / VAC^4)

Below, I've listed the vacuum wavelength of some transitions:

 Air       Vacuum
 --------  --------
 4861.363  4862.721  H_beta
 4958.911  4960.295  [O III]
 5006.843  5008.239  [O III]
 6548.05   6549.86   [N II]
 6562.801  6564.614  H_alpha
 6583.45   6585.27   [N II]
 6716.44   6718.29   [S II]
 6730.82   6732.68   [S II]

Note that the wavelengths are also shifted such that measured velocities will be relative to barycentric at the mid-point of each 15-minute exposure (using TAI-BEG + 0.5 * EXPTIME from the header).

Problems

Known Problems

Here is a list of the most offensive known problems in either the data or in the Spectro-2D reduction code as of version v4_9_11 (31 May 2002):

Plate 0187/51455 has no good b2 arc on the first night of data 51454.
                 This plate has severe scattered light problems.  The r2 camera
                 is completely flagged as REDMONSTER in the ORMASK, and is
                 therefore ignored by the redshift-finding code.  For that
                 reason, the synthetic spectra do not match the spectra
                 at all for the red side of fibers 321-640.
Plate 0192/51461 has severe scattered light problems.
Plate 0202/51441 has the Red Monster light at 6400-6550 Ang in the last
                 2 of 4 exposures (exp #1367,1368).
                 This results in some wrong z=4.3 QSOs (fibers 601,639).
                 Apparently 2 of the b1 science exposures saturated
                 (exp #1366,1367).  See PR #1890.
Plate 0214/51441 has the Red Monster light at 6400-6550 Ang in the first
                 3 of 4 exposures (exp #1356-1368).  See PR #1890.
Plate 0231/51456 has no good r1 arc on the first night of data 51455.
Plate 0260/51612 looks like funny fluxing in the blue below 4000 Ang,
                 probably due to very low S/N.
Plate 0354/51792 taken during moon
Plate 0360/51780 had the b1 CCD warm up during exposures 6193,6194,6199
                 resulting in apparent scattered light in 6193,6194 and
                 rejection of frame 6199.  See PR #2170.
Plate 0426/51882 has the Red Monster light at 6400-6550 Ang in exp #7366-7367.
                 See PR #1890.
Plate 0541/51959 has bad extraction chi^2 and sky-subtraction, especially in r1.
Plate 0644/52149 has bad extraction chi^2 and sky-subtraction.  Is this the same
                 problem as for plate 0541/51959?
The following list of plates were rejected because of out-of-collimation spectrographs (c.f., sdss-spectro/873):
Plate 0504/52316 - Many rejected pixels
Plate 0704/52205 - Many rejected pixels
Plate 0721/52228 - Many rejected pixels
Plate 0736/52210 - Many rejected pixels
Plate 0756/52237 - Badly out-of-focus
Plate 0761/52266 - Many rejected pixels
Plate 0769/52282 - Many rejected pixels
Plate 0770/52282 - Many rejected pixels
Plate 0775/52295 - Many rejected pixels
Plate 0778/52337 - Many rejected pixels
Plate 0790/52346 - Badly out-of-focus night
Plate 0791/52347 - Badly out-of-focus
Plate 0873/52347 - Badly out-of-focus
Plate 0876/52346 - Badly out-of-focus night
Plate 0889/52346 - Badly out-of-focus night; many rejected pixels

Failed Plates

Plates 0179,0182 are FASTT plates with 12th mag stars and no sky fibers;
                 these data are not reducable.
Plate 0204/51455 has no good b1,b2,r2 arcs
Plate 0209/51462 has too many saturated rows in r2 for the modern reduction code
Plate 0213/51578 has no good b2 arc
Plate 0218/51461 has no good b2 arc
Plate 0235/51462 has no good b2 arc (saturated)
Plate 0260/51612 has only 2 science exposures, so will not combine
Plate 0284/51934 has no good arcs (flat-field screen was not closed for arcs);
                 no plan file even built
Plate 0296/51578 has no good b2 arc
Plate 0317/51589 has no flats or arcs; no plan file even built
Plate 0318 special plate needs edited plug-map files in order to run
Plate 0324/51616 only has one science exposure, so won't combine
Plate 0347/51701 has 1 saturated exposure on 51694, and 1 good one on 51701
Plate 0560/52045 only has one science exposure, so will not combine

Dates and Plates to Remember

MJD -> 51569 (->25 Jan 2000): Shutter open during readout!
                   Definitely compromises data for plates 187,188,190,191,
                   192,194,198,202,203,217,231.
MJD -> 51605 (->2 Mar 2000): Scattered light from internal LED's contaminate
                   all these data.  After this date, we went through several
                   work-arounds that basically eliminated the problem.
                   Definitely compromises data for plates 187,188,190,191,
                   192,194,198,202,203,214,217,231,265,266,269,283,295.
MJD 51578 - 55580 (3-5 Feb 2000): Severe "shifted row" electronics problem.
                   This only effects plate 295/51580.
MJD 51584,51585,51608,51614,51615,51629,51630: Raw images are the wrong size!
                   Unsure what this actually does to the data.
                   Potentially effects plates 266,267,279,281,282,283,295,303.
MJD 51688 - 51703 (22 May 2000): Electronics "dropped pixel" problem
                   on cameras b2,r2.  Potentially effects plates 312,338,339,
                   343,344,346,348,349,351,353.
MJD 51779 only (22 Aug 2000): Terrible electronics problem on r2.
                   This only effects plate 356/51779.
MJD 51789 - today: (2 Sep 2000): Smear implemented for most plates.
MJD 52220 - 52266, 52312 - 52356 (Nov-Dec 2001, Feb-Mar 2002) Spectrograph
                  slit-head latch problems.  The kinematic mounts for the
                  slit-heads were not always engaged, resulting in out-of-focus
                  exposures on the instrument.  The optimal extraction code
                  works quite poorly on these data, as does the sky-subtraction
                  which shows large relative chi^2 (MJD 52346 is the worst,
                  plates 790, 876, 889).
MJD 52899 - today: Noise source in right amplifier of r2 adding ~30 DN of noise
                  to ~5% of the pixels.  See sdss-obs/4425 and other e-mail
                  traffic after 20 March 2004.
MJD 52988,52991: LED from telephone causing red spotlight on plate 1489/52991.
                  See sdss-spectro/1245 and sdss-spectro/1246.
MJD 53053 (18 Feb 2004): A fiber bundle on cartridge #9 spectro-1 detached
                  while observing plate 1372.
MJD 53670-53676 (26-28 Oct 2005): Condensation inside the b1 camera
                  make the data useless on plates 2264/53673, 2333/53676,
                  2338/53676 (c.f. PR #6713).  The data on 2265/53674 and
                  2266/53675 appears surprisingly OK.

Plates   1 - 431: Tiling bug resulted in larger color range for SPECTROPHOTO
                  standards, which must compromise spectro-photometry
                  at some level.
Plates 371 - 416: Bad QSO selection due to bug in target v2_7 (chunk 8),
                  resulting in a very low success rate for quasars.

How to Report Problems

Problems can be reported via the bug database at:


http://spectro.princeton.edu/cgi-bin/gnatsweb.pl
File bugs about the extracted spectra against the category "idlspec2d", and about the redshifts or template-fitting against "specBS". Before filing a problem report (PR), first look at the Known Problems section above and search the bug database to make sure no one else has already reported the problem.

Acknowledgements

Funding for the creation and distribution of the SDSS Archive has been provided by the Alfred P. Sloan Foundation, the Participating Institutions, the National Aeronautics and Space Administration, the National Science Foundation, the U.S. Department of Energy, the Japanese Monbukagakusho, and the Max Planck Society. The SDSS Web site is http://www.sdss.org/.

The SDSS is managed by the Astrophysical Research Consortium (ARC) for the Participating Institutions. The Participating Institutions are The University of Chicago, Fermilab, the Institute for Advanced Study, the Japan Participation Group, The Johns Hopkins University, Los Alamos National Laboratory, the Max-Planck-Institute for Astronomy (MPIA), the Max-Planck-Institute for Astrophysics (MPA), New Mexico State University, Princeton University, the United States Naval Observatory, and the University of Washington.


David Schlegel, schlegel@astro.princeton.edu