Source and Feature Detection

A Working Group in Astrostatistics

Group Leaders: David van Dyk (leader), Zhengyuan Zhu (alternate), and Lingsong Zhang (web)


Announcements:

WE WILL HAVE TELECONFERENCE ON MAY 22!

Please check my temporary website for instant updates.

Since I can not update the SAMSI website at home, I setup a temporary website for the group. I will update all the information of the SAMSI website every Monday morning. Please also check the temporary website at

http://www.unc.edu/~lszhang/samsi-sd/

Old pdf files and data sets are available just at SAMSI website

Meeting Time: Monday 1:00 pm - 4:30 pm (EST)
Location: NISS building, 2nd floor classroom 203

Remote access for the meeting is available:

  • Via teleconference at 919-685-9338 not earlier than the scheduled teleconference time.
  • Webcam will not be available this semester. Sorry about that!

Note: For audio, participants will have to call into the conference line this semester.


Data Sets and Other Resourses

Password Protected Site (Data Files and other protected documents)

Meeting Activities

Date Topics & Readings Notes
May 22, 2006

2:30: Group 1 (Modeling High-Energy Count Data)

* Analysis of Jim's "galaxy data"
* Analysis of Becca's test data
* Final Report & Future Work

3:00: Group 2 (Solar Image Analysis)

* Final Report & Future Work

3:30: Gropu 3 (Upper Limits)

* Final Report & Future Work


Some documents are availale at the secure website: for-jim.pdf.gz, for-jim2.pdf.gz, for-becca.pdf.gz
Some pictures from Thomas Lee (Trace image Viney mentioned) t1.pdf, t2.pdf, t3.pdf, t4.pdf


On the phone: David van Dyk, Alex Young, Viney Kashyap, Jim Chiang, Alanna Connors, Aneta Siemiginowska, Ji Meng Loh, Yaming Yu, Jogesh Babu etc.

In the room: Zhengyuan Zhu, Lingsong Zhang

May 15, 2006

No meeting unless someone set it up!

 
May 8, 2006

Group 3: Statistical Issues with Low-Count Data

1:40. Topic 3a: Upper Limits
* Any updates?

Group 1: Modeling High-Energy Count Data

1:50. Topic 1a: Glast Data
* Any updates?

Gropu 2: High Resolution Image Analysis

2:00. Solar Image Analysis
Preliminary results from Thomas Lee (alpha, beta, betagamma, betagammadelta)
Lingsong's FDA exploration on Solar Images (eit-195) .pdf (if time permits)

On the phone: David van Dyk, Viney Kashyap, Jim Chiang, Alanna Connors, Aneta Siemiginowska, Thomas Lee etc.

In the room: Zhengyuan Zhu, Jogesh Babu, Lingsong Zhang etc.

May 1, 2006

Group 3: Statistical Issues with Low-Count Data

2:30. Topic 3a: Upper Limits
* Recap
* Discuss Potential Survey Data
* Preliminary results (27_psfsiz.pdf)

Group 1: Modeling High-Energy Count Data

3:00. Topic 1a: Glast Data
* Status of new EMC2 code
* Prior specification in MRF (results_mfr_pr_sens.tar.gz)
* Prior specification in MS (results_ms_pr_sens.tar.gz)
* Maps, Means, and Simulations (results_ms100.tar.gz)
* Posterior simulations and error bars (This goes into topic 4.)
* Alanna's data
* cfitsio: Can someone at CfA help?


Jim's email on April 30.

On the phone: Viney Kashyap, Andreas Zezas, Alanna Connors, David van Dyk. Jim Chiang etc.

In the room: Jogesh Babu, Lingsong Zhang, Zhengyuan Zhu.

April 24, 2006

Today's teleconference is cancelled!

2:30 pm: We will have a short meeting on Monday to discuss survey data for an upper limit analysis and to give an update on the EMC2 code.

 
April 17, 2006

Group 1: Modeling High-Energy Count Data

2:30. Topics 1a & 1b: GLAST Data and Solar RHESSI Data
* Preliminary EMC2 Results
* Preliminary MRF Results
* Discuss edge effects & how to customize the model
* Solicit test data from Alanna and Alex

Some preliminary plots and related data sets from David, Check the Password Protected Website

Jim's preliminary plots, Check at ftp://ftp-glast.slac.stanford.edu/glast.u06/jchiang/SAMSI/*.fits

On the phone: Aneta Siemiginowska, Alanna Connors, Jim Chiang, Alex Young, David van Dyk, Yaming Yu, etc.

In the room: Lingsong Zhang, Hyunsook Lee, Zhengyuan Zhu, Ji Meng Loh.

April 10, 2006

2:00 pm

"Multiscale Analysis of Photon-Limited Astronomical Signals and Images" by Rebecca Willett,
(Abstract .txt)

Contact the speaker for the slides.

On the phone: Vinay Kashyap, Thomas Lee, Alanna Connors, David van Dyk, Taeyoung Park, Jim Chiang, Alex Young etc.

In the room: Zhengyuan Zhu, Ji Meng Loh, Lingsong Zhang, Hyunsook Lee, etc.

April 3, 2006

Group 3: Statistical Issues with Low-Count Data

2:15. Topic 3a: Upper Limits
* Multiplicitve Source Intensities (see .pdf)
* Discuss Vocabulary
* Isolate a wroking example

Group 2: Modeling Optical and Solar Images

3:00. Topic 2a: Solar Astrophysics
* Update
* Morphological Analysis


Thomas' results. (alpha, beta, betagamma, betagammdelta)

On the phone: David van Dyk, Alanna Connors, Keith Arnaud, Jim Chiang, Yaming Yu, Aneta Siemiginowska, Viney Kashyap, Thomas Lee, Rebacca Willett, Taeyoung Park, Mark Weber
etc.

In the room: Lingsong Zhang, Jogesh Babu

March 27, 2006

2:00 pm - 4:00 pm

"A Geometric Approach to Distinguish Between a New Source and Random Fluctuations" by Ramani S. Pilla (Abstract .pdf)

Contact the speaker for the slides if you need them!


Related resources:

Pilla, Loader & Taylor (2005) Physical Review Letters article & more on applications at http://stat.case.edu/~pillar/PRL/PRL.htm

Pilla & Loader (2003) Technical Report/E-print Archive at
http://arxiv.org/abs/math.ST/0511503

Another paper related to parameter confidence (mentioned by Aneta on March 20)

Yaqoob, Tahir (1998), Parameter Estimation in X-Ray Astronomy Revisited, The Astrophysical Journal, 500, pp 893-898 .pdf

On the phone: Jim Chiang, Rebacca Willett, Alex Young, Alanna Connors, Yaming Yu, Xiao-Li Meng, Taeyoung Park, Aneta Siemiginowska, Vinay Kashyap, Jingchen Liu
etc.

In the room: Ramani Pilla, Lingsong Zhang, David van Dyk, Ji Meng Loh, Bill Jefferys, Jogesh Babu, M. Woodroofe, Louis Lyons, Zhengyuan Zhu, Tom Loredo etc.

March 20, 2006

Group 1: Modeling High-Energy Count Data

2:00. Topic IA, Image Reconstruction for GLAST
* Jim's preliminary results using Nowak and Kolaczyk multiscale model (slides .pdf)

Group 3: Statistical Issues with Low-Count Data

2:40. Topic 3A Upper Limits
* DvD's updated simplest case results (.pdf) Using the power-based method
* Conditioning on the Ancillary Total Count
* Consensus terminology
* Outline ideas for technical report

3:20. Topic 3B Poisson Data in the Presence of Background
READING GROUP. We will discuss: "Determination of Confidence Limits for Experiments with Low Numbers of Counts" (Kraft, Burrows, and Nousek, 1991, ApJ, 374, 344-355, .pdf)

On the phone: Alex Young, Jim Chiang, David van Dyk, Alanna Connors, Rebecca Willett, Aneta Siemiginowska, Viney Kashyap, Peter Freeman

In the room: Ji Meng Loh, Lingsong Zhang, Zhengyuan Zhu

  About Upper Limit: Yaming's comments .pdf. Zhengyuan's Comments .pdf  
March 6, 2006

Group 1: Modeling High-Energy Count Data

2:50. Topic IA, Image Reconstruction for GLAST
* No Preliminary Results Yet....

Group 3: Statistical Issues with Low-Count Data

3:00. Topic 3A Upper Limits
* Vinay's new terminology
* DvD's results for Vinay's simplest cases
* Can we apply to a survey of data?
This will test the frequency properties.
Do we get too many empty intervals like the physicists

3:30. Topic 3B Poisson Data in the Presence of Background
READING GROUP. We will discuss: "Confidence Limits for Small Numbers of Events in Astrophysical Data" by N Gehrels (1986), 303, 335-346. (.pdf is on webpage)

David: * I think this is a pretty basic paper. So there may not be too much to discuss! But we should start from the beginning!


Some papers to read:

1. Gehrels, Neil, (1986), Confidence Limits for Small numbers of events in Astrophysical Data, The Astrophysical Journal, 303, pp 336-346. .pdf

2. Kraft, R. P., Burrows, D. N. and Nousek, J. A., (1991), Determination of Confidence Limits for Experiments with Low Numbers of Counts, The Astrophysical Journal, 374, pp 344-355. .pdf

3. Ptak, A. et al. (2006), Type II Quasars from the Sloan Digital Sky Survey. IV. CHANDRA and XMM-NEWTON observations reveal Heavily Absorbed Sources, The Astrophysical Journal, 637, pp 147-156. .pdf

4. Miscellanea, (Fiducial Limits for the Poisson Distribution, by F. Garwood), .pdf

Viney's description about Upper Limit .pdf.

David's preliminary result of Viney's simplest cases. .pdf

Another paper mentioned by Alex: Cherry, M. L. et al, (1980), Statistical Evaluation of Gamma-Ray Line Observations, The Astrophysical Journal, 242, pp 1257-1265. .pdf

On the phone: Jim Chiang, Alex Young, Viney Kashyap, Keith Arnaud, Andreas Zezas, Ruth Barrera, Antonio Uribe

In the room: Bill Jefferys, Zhengyuan Zhu, Ji Meng Loh, David van Dyk, Lingsong Zhang

Feb 27, 2006

Group 3: Statistical Issues with Low-Count Data

1:30. Topic 3B Poisson Data in the Presence of Background
READING GROUP. We will discuss: "Setting Confidence Intervals on Bounded Parameters" by M. Mandelkern, Statistical Science (2002), 17, 149-172.

Group 2: Modeling Optical and Solar Images

2:00: Topic 2a: Solar Astrophysics
* Preliminary Results (?)

Group 3: Statistical Issues with Low-Count Data

2:30. Topic 3A Upper Limits
* DvD's simplest case results
* Vinay's list of simple cases!


David's rusult. (.pdf)
Vinay's result. (.pdf)
Here are some more numbers, courtesy Frank Primini, who calculated the bounds as suggested by Feldman & Cousins (.pdf)

Alex's Email on Feb 27 .pdf
Thomas' Email on Feb 27 .pdf
Thomas' preliminary results (alpha.pdf, beta.pdf, betagamma.pdf, betagammadelta.pdf)

Eric's materials about Small N problems is in the secure data directory. If you did not get it from the email list, and you would like to view it, please contact Lingsong Zhang for the password. Get the paper (only those in the SAMSI program can get the password, sorry for the rest)

On the phone: David van Dyk, Alanna Connors, Vinay Kashyap, Alex Young, Frank Primini, Jeff Scargle, Rebecca Willett, Thomas Lee, Eric Feigelson, Yaming Yu

In the room: Jogesh Babu, Ji Meng Loh, Woncheol Jang, Hyunsook Lee, Lingsong Zhang, Zhengyuan Zhu, Bill Jefferys

Feb 20, 2006

Group 1: Modeling High-Energy Count Data

1:45. Topic IA, Image Reconstruction for GLAST !!!Postponed to next week!!!
* Data Questions
* Preliminary Results (?)


2:00. "A Fully Bayesian Poisson Image Reconstruction Technique for High-Energy Astrophysics", David van Dyk, This is his talk about EMC2.

Group 3 topics will be postponed until 2/27


David's slides .pdf

On the phone: Alanna Connors, Jim Chiang

In the room: David van Dyk, Lingsong Zhang, Ji Meng Loh, Zhengyuan Zhu, Bill Jefferys, Hyunsook Lee

 

Alanna's Email about Measuring What We "See" in Poisson Images, or, Is That a Bridge over the Milky Way? .pdf

Alanna's questions about non-parametric blob-detection. .pdf

Woncheol's response to Alanna's questions about Non-parametric Blob-detection: .pdf
Related pdf files: Cuevas, Febrero and Fraiman (2000), .pdf Devroye and Wise (1980) .pdf Cuevas and Fraiman (1997) .pdf Jang (2006) .pdf Polonik (1995) .pdf Wong and Moore (2002) .pdf

 
Feb 13, 2006 Group 1: Modeling High-Energy Count Data
1:00: Topic 1a: Image Reconstruction for GLAST
* Update on status of EMC2 code (David)
* Status of test data and psf
1:20: Topic 1b: Image Reconstruction for RHESSI
* The BIG question: Can we get IDL code for R-L?
* Status of test data and psf
* Can we use EMC2 for this data?

Group 3: Statistical Issues with Low-Count Data
1:40: Topic 3b Poisson Data in the Prsence of Background
* Literature review?
Interested readers might try to have a look at:
"Setting Confidence Intervals on Bounded Parameters" by M. Mandelkern, Statistical Science (2002), 17, 149-172. Available on J-stor.

Group 2: Modeling Optical and Solar Images
2:00: Topic 2a: Solar Astrophysics


David suggests: Interested readers might try to have a look at: M. Mandelkern (2002), "Setting Confidence Intervals on Bounded Parameters" , Statistical Science, 17, 149-172. Available on J-stor. .pdf Full Version paper with discussions and rejoinder .pdf

Alex's slides (his website, .pdf); His email on Feb 13, .pdf
Note: his website contains some image data sets

Jeff suggests:

Pilla, Loader & Taylor (2005) Physical Review Letters article & more on applications at http://stat.case.edu/~pillar/PRL/PRL.htm (This is also mentioned by Aneta on Feb 06)
Pilla & Loader (2003) Technical Report/E-print Archive at
http://arxiv.org/abs/math.ST/0511503, .pdf at SAMSI
Jim's comments .pdf
Peter's comments .pdf Related paper: Freeman et. al (1999), Statistical Analysis of Spectral Line Candidates in Gamma-Ray Burst GRB 870303, The AstroPhysical Journal, 524, pp 753-771..pdf at SAMSI.

Alanna's notes for today, thanks! (web link, SAMSI backup)

Jim placed some simulated LAT data at ftp://ftp-glast.slac.stanford.edu/glast.u06/jchiang/SAMSI/ cmap_highE.dat are pixel counts for a 128x128 image and cmap_psf.dat is the point spread function sampled at the same resolution, but covering a smaller region, 40x40. Both arrays have been flattened row-wise, but dealing with the transpose should work just as well. cygnus.png is what the original fits file looks like when displayed with ds9. (Jim's Email .pdf)


On the phone: Alex Young, Jim Chiang, Aneta Siemiginowska, Keith Arnaud, Alanna Connors, Thomas Lee, Rebacca Willett

In the room: David van Dyk, Jogesh Babu, Lingsong Zhang, Ji Meng Loh, Zhengyuan Zhu, Hyunsook Lee

Feb 06, 2006

2:00-4:00 EST

Teleconference meeting agenda:

Group 1: Modeling High Energy Counts
2:00 Topic 1A: Image Reconstruction for GLAST
2:30 Topic 1B: Image Reconstruction for RHESSI

Group 3: Statistical Issues with Low-Count Data

3:00 Topic 3A Upper Limits
3:30 Topic 3B Poisson Data in the Presence of Background


David's Email on Feb 2. Topics of working groups .pdf

Aneta's Slides .pdf
Jim's questions .pdf
Alex's Slides .pdf
Eric's bibliography .pdf
Vinay's Slides .pdf

Aneta suggests: Pilla, R., Loader, C., and Taylor, C. (2005).  New technique for finding needles in haystacks: Geometric approach to distinguishing between a new source and random fluctuations. Physics Review Letters, 95, 230202. Information available at
 http://stat.case.edu/%7Epillar/PRL/PRL.htm

Alanna's informal draft notes of Monday's meeting (Alanna's link, SAMSI backup)

Alanna's questions (Alanna's link, SAMSI backup)

Rebacca suggests: G. Kerkyacharian, D. Picard, and K. Tribouley, lp adaptive density estimation, Bernoulli, vol. 2, pp. 229?247, 1996. .pdf There was also some talk of online software for the Nowak-Kolaczyk work. I have written code for Poisson intensity estimation based on my extensions of Nowak and Kolaczyk's work; it is available at http://www.ee.duke.edu/~willett/Research/software.html.

On the phone: Jim Chiang, Alanna Connors, Vinay Kashyap, Taeyoung Park, Jeff Scargle, David van Dyk, Alex Young, Rebecca Willett, Aneta Siemiginowska, Eric Feigelson, Keith Arnaud, Peter Freeman

In the room: Jogesh Babu, Ji Meng Loh, Woncheol Jang, Hyunsook Lee, Lingsong Zhang, Zhengyuan Zhu

 

If you are there, but not in the list, you could contact me. I will add you into the list.

Jan 30, 2006 David's Email on Jan 27  

Jan 26, 2006

Kickoff Workshop

Form subgroups, develop plans for the semester.

Agenda .pdf

Peter Freeman, "Feature Detection: in the trenches"

Alanna's presentation "CHALLENGE: “BLOB DETECTION”?" .pdf

 

More information will be added later.

For those who presented, please send us the slides. Thanks.


Subgroups

High-Energy Modeling Solar/Optical Modeling Upper Limits Image Error Bars Statistical Computing
Yaming Yu Yaming Yu Yaming Yu Adam Roy Xiao-Li Meng
Adam Roy Hyunsook Lee Aneta Siemiginowska Xiao-Li Meng Thomas Lee
Aneta Siemiginowska Vinay Kashyap Jim Chiang Thomas Lee Taeyoung Park
Jim Chiang Thomas Lee Hyunsook Lee Aneta Siemiginowska Keith Arnaud
Jeff Scargle Alex Young Jiashun Jin Jim Chiang Peter Freeman
Jiashun Jin Aneta Siemiginowska Woncheol Jang Jeff Scargle Alex Young
Vinay Kashyap Rebecca Willett Viney Kashyap Hyunsook Lee Vinay Kashyap
Taeyoung Park Lingsong Zhang Ji Meng Loh Jiashun Jin Aneta Siemiginowska
Peter Freeman   Taeyoung Park Woncheol Jang Zhengyuan Zhu
Alex Young   Xiao-Li Meng Vinay Kashyap David van Dyk
Keith Arnaud   Peter Freeman Ji Meng Loh Lingsong Zhang
Alanna Connors   Alex Young Peter Freeman  
David van Dyk   Rebecca Willett Alex Young  
    David van Dyk Keith Arnaud  
    Lingsong Zhang Alanna Connors  
    Jogesh Babu David van Dyk  

Group Members

Names Affiliation Email
S Woncheol Jang Duke University [email protected]
A Vinay Kashyap Smithsonian Astrophysical Observatory [email protected]
S Lingsong Zhang University of North Carolina [email protected]
S Jiashun Jin Purdue University [email protected]
A Keith Arnaud NASA Goddard Space Flight Center [email protected]
A Alanna Connors Eureka Scientific [email protected]
A Jeff Scargle NASA Ames Research Center [email protected]
S Hyunsook Lee Pennsylvania State University [email protected]
S Yaming Yu University of California, Irvine [email protected]
S Taeyoung Park Havard University [email protected]
S Xiao-Li Meng Havard University [email protected]
S Thomas Lee Colorado State University [email protected]
A Jim Chiang GLAST [email protected]
S Adam Roy University of California, Irvine [email protected]
A Peter Freeman Carnegie Mellon University [email protected]
S Chris Genovese Carnegie Mellon University [email protected]
S David van Dyk University of California, Irvine [email protected]
S Rebecca Willett Duke University [email protected]
A C. Alex Young NASA Goddard Space Flight Center [email protected]
A Aneta Siemiginowska Smithsonian Astrophysical Observatory [email protected]
A Martin Weinberg University of Massachusetts [email protected]
S Ji Meng Loh Columbia University [email protected]
S Jayanta Pal University of Michigan [email protected]
S Zhengyuan Zhu University of North Carolina [email protected]
S Angela Hugeback University of Chicago [email protected]
A Jack Ireland NASA [email protected]
S G. Jogesh Babu Pennsylvania State University [email protected]
A Eric D. Feigelson Pennsylvania State University [email protected]
A Frank Primini Smithsonian Astrophysical Observatory [email protected]
S Ruth Stella Barrera Rojas
National University of Colombia [email protected]
A Andreas Zezas
Smithsonian Astrophysical Observatory [email protected]
A Johann Cohen-Tanugi GLAST [email protected]

If you are interested in participating in a working group, please send e-mail to [email protected] giving your full contact information and stating which working group(s). You will be sent instructions by e-mail about how to participate.

Send your comments about this site to Lingsong Zhang ([email protected]).

To subscribe or unsubscribe to the email list, contact Lingsong Zhang ([email protected])