Blog

Artemis II Solar Eclipse

2026-04-09T00:00:00+00:00

The Artemis II mission launched on the 1st April 2026; this launch date allowed the crew to observe a solar eclipse on the 6th April(EDT) / 7th April (UTC) after transiting the far side of the moon.

Image credit NASA

We on the SunPy blog rarely miss the opportunity to talk about a solar eclipse. So when we saw the stunning photos taken by the astronauts on Artemis II, we wanted to use SunPy to compare them to other photos of the solar corona. I do highly recommend watching the recording of the eclipse on YouTube; the reactions and descriptions of the astronauts are worth it.

Amongst the many amazing photos downlinked during the mission was this image of the solar eclipse:

Image credit NASA

This image is particularly good for comparing to other solar data because the limb of the moon is clearly visible, and there are stars and planets in the image we can use as references. These features will allow us to determine exactly where and at what angle the camera was pointing. At the end of the post you will be able to see how we can overlay on this photo images taken by solar observing satellites.

Fitting Coordinate Information

To be able to compare this image with other observations of the Sun, we need to identify where the camera was pointed and how it was rotated. To do this we perform the following steps:

Extract the time information from the metadata stored in the image.
Use the time information to lookup the exact position of Artemis II.
Fit the edge of the moon to identify the location of the center of the moon, and the size of the moon in the image.
Use the three planets visible in the lower right of the image to identify the rotation angle.
Use the planets to fit the distortion of the lens.

All the code for this example is in The sunpy Gallery.

Finding the position of Artemis II

The first step is to know the time the image is taken; we can extract this from the EXIF metadata. Once we have this we query JPL Horizons for the position of Artemis II.

from sunpy.coordinates import get_horizons_coord

artemis2_naif_id = "-1024"
artemis2_coord = get_horizons_coord(artemis2_naif_id , "2026-04-07 01:06:19")

We can also use the positions returned by JPL Horizons and the coordinates packages in sunpy and astropy to visualize what part of the Artemis II trajectory was in eclipse. To see the details of how this was done see this example in the SunPy gallery.

Visualization of the Artemis II trajectory with the eclipse highlighted.

Moon Limb Fitting

The next step is to find a known location in the image, a reference point. The easiest one for us to use is the center of the moon, which we find by doing edge detection and Hough filtering, using scikit-image.

import numpy as np

from skimage.feature import canny, peak_local_max
from skimage.transform import hough_circle, hough_circle_peaks

edges = canny(eclipse_image, sigma=2)

h, w = eclipse_image.shape
radii = np.arange(0.25*h, 0.4*h, 10)

hough_res = hough_circle(edges, radii)
accums, cx, cy, rad = hough_circle_peaks(hough_res, radii, total_num_peaks=1)

A cropped view of the Moon, showing the results of the canny edge detection algorithm and the Hough filter.

Calculating Image Scale

Based on the determined center of the moon is and its radius in the image we can construct a coordinate system for the image.

from astropy.coordinates import SkyCoord
import astropy.units as u

moon = SkyCoord(coords["moon"], observer=coords["artemis_ii"])
R_moon = 0.2725076 *  u.R_earth  # IAU mean radius
dist_moon = SkyCoord(coords["artemis_ii"]).separation_3d(moon)

moon_angular_width = np.arcsin(R_moon / dist_moon).to(u.arcsec)
im_radius = rad * u.pix
plate_scale = moon_angular_width / im_radius

Using this information we can build a sunpy map (see the gallery example for details). Plotting this alongside the locations of the planets results in:

Initial coordinate system fit to image, notice that the locations of the highlighted planets are incorrect.

Fitting Roll Angle

It’s clear from the previous image that the image is rotated around the center of the moon. We can solve for this rotation by using a peak finding algorithm to locate the planets in the image and comparing these positions to the planets coordinates extracted from JPL Horizons. Doing this results in a \(-21.2^\circ\) roll angle which we can add to our Maps metadata.

Image showing the expected positions of the planets and the detected (peaks) positions of the planets, from which the roll angle is calculated.

Fitting Lens Distortion

The final correction to apply to our fitted coordinate system is the distortion of the camera lens (a Nikkor AF 135mm f/2D DC). This makes objects distant from the centre of the image appear even more distant than they should. We can quantify exactly how much the image has been distorted through comparing the expected vs actual positions of Mars and Mercury (not Saturn as it is too close to the center of the image). We add this distortion to our coordinate system and our planets now appear in the correct place.

Coordinate system fit to with additional correction for lens distortion.

Overplotting Coronagraph Images

Now that we have fit a coordinate system to the eclipse photo we can compare this observation of the corona to other data. To do this we are going to use the coronagraph on the Solar and Heliospheric Observatory (SOHO) which is located around the Sun-Earth L1 point. We reproject (or re-grid) these images to the fitted coordinate system of the Artemis II image to compensate for differences in satellite location and point of view, and then crop them to the limb of the moon.

The Artemis II solar eclipse photo with the positions of Mercury, Mars and Saturn highlighted, and coronagraph images from SOHO’s LASCO instrument plotted over the disc of the moon.

We hope you have found this post interesting. The full code for this post can be found in The sunpy Gallery. Remember, that if you are lucky enough to observe the total solar eclipse which will be visible from parts of Europe in August 2026 and you take a photo, you can try this type of analysis with your own photos, by following our previous blog post!

Anaconda packages are not “free”

2024-08-09T00:00:00+00:00

We wanted to inform the wider community about Anaconda Inc. and if you are legally allowed to use their products for free.

What is not commonly known is that Anaconda Inc. has implemented specific clauses in its software licenses which determine if one is able to use Anaconda or Anaconda Navigator for free, based on the organization you work for. For example, The Register has reported that:

Research and academic organizations are just now finding out that they will have to pay for software made by Anaconda, when for years these groups were under the impression it could be used at no cost.

A source who works at a medium-size non-profit academic research institution told The Register about being on the end of a legal demand to purchase a commercial license for the Anaconda-built software they had been using for free.

Anaconda themselves summarize this in a really detailed blog post, which we use here to form the basis of this post.

Glossary

I want to define a few words which will be used heavily below:

conda: This refers to the binary tool that a user calls on the command line. It is a package installation and environment management software.
channel or conda channel: It is a centralized repository of packages (python or not) which one can install with conda.
distribution: It is an installer that will install conda and configure a default channel to use. For anaconda, this is the “defaults” channel whereas for conda-forge this is the “conda-forge “ channel.

What is free to use?

conda binary.
mamba an alternative to conda.
Anything that is provided by the conda-forge project which means their packages, the channels they maintain and any distribution they provide.

What is not free to use?

Anaconda’s “defaults” channel, which is used for the base environment. It is used if one installs the Anaconda distribution or the miniconda distribution. Anything that is “curated, built, maintained, and served by Anaconda’s engineers on its secure cloud infrastructure”, has these licenses.
Anaconda Navigator which is a Graphical User Interface (GUI) that many people use to manage their Anaconda install.

Using the defaults channel or Anaconda Navigator can lead to you becoming legally required to pay Anaconda Inc. for their use.

If you want to use these for free, you have to meet the following conditions:

Your organization has less than 200 people, or
Your organization has 200 or more people, but qualifies as an exempt organization in Anaconda’s terms of service:

Students and educational entities may use our free offerings in curriculum-based courses.

Following the publication of the Register article, the CEO of Anaconda, Peter Wang posted on Linkedin. He says that:

I want to be very clear: Anaconda’s installers & package repos are free for teaching, learning, and research at accredited educational institutions worldwide.

and

Our legal team is taking a comprehensive look at the wording in our current ToS, EULA, and related documents, with a focus on clarity for academic and academic research use.

and

In addition, we’re looking into a flexible pricing structure for non-profit organizations with over 200 full-time employees

Hopefully, by the end of the year with these changes in place, users will have a clear understanding if they are allowed to use Anaconda Inc. products for free.

What is the alternative?

The answer from the SunPy Project is the following:

Use the “conda-forge” channel provided by the conda-forge project. conda-forge is a community-led project which creates recipes, hosts infrastructure and distributions for use with conda. They provide a distribution (similar to miniconda) called miniforge is configured to use the “conda-forge” channel by default. It is also the only way to install any SunPy Project library via conda.
Never setup or use the “defaults” channel if you install miniforge.

A blog post written by Tim de Jager & Ruben Arts summarizes how conda-forge is free and avoids any vendor lock in.

I have Anaconda or Anaconda Navigator already - What should I do?

Though, you could get rid of the default channel(s) using the conda config command as shown in this Stack Overflow answer about switching channels from anaconda to conda-forge. This is good for new environments but it doesn’t remove what you’ve got already installed (e.g., the base environment) and you may be still infringing the Terms of Services for Anaconda. If you have Anaconda Navigator, you have to remove it unless you know that you are not violating the license requirements.

Therefore, the cleanest method is to completely remove Anaconda and Anaconda Navigator and install miniforge. Unfortunately, there is no automated way of recreating all of your environments, and this post about migrating from Anaconda to miniforge details the steps that one has to follow to migrate.

SunPy Development Team wins a NASA Group Achievement Award!

2024-06-19T00:00:00+00:00

We are delighted to announce that the SunPy development team has been awarded with a NASA Group Achievement Award!

The award was given to our team for “developing and continually improving free and reliable open-source software that supports NASA missions and the analysis of NASA’s vast archives of solar physics data”.

This award is a collective achievement, and we share it with everyone who has contributed to SunPy! We look forward to continuing our mission of supporting solar physics research and making impactful contributions to the scientific community.

Process Your Solar Eclipse Photos with SunPy!

2024-04-03T00:00:00+00:00

This blog post was written in a Jupyter notebook. Click here for an interactive version:

On April 8th, 2024, a total solar eclipse will pass over North America. A total solar eclipse happens when the Moon passes between the Sun and Earth, completely blocking the face of the Sun. Only during totality, when the bright disk is completely obscured, is it possible to see with the naked eye the solar corona, the outermost layer of the Sun’s atmosphere. The total solar eclipse will give millions across North America the chance to see and photograph the solar corona.

In this blog post, we will show how you can use SunPy to process your photos of the eclipse. To do this, we will use an image from the 2017 solar eclipse that also passed over North America, the so-called “Great American Eclipse”. We will walk through processing this image with SunPy as well as other Python libraries, so that you can generate a coordinate system for your image. As we will show, this allows you to combine your eclipse images with solar observations such as those from NASA’s Solar Dynamics Observatory.

[1]:

from pathlib import Path

import astropy.units as u
import exifread
import matplotlib.image
import matplotlib.pyplot as plt
import numpy as np
import sunpy.coordinates
import sunpy.coordinates.sun
from astropy.constants import R_earth
from astropy.coordinates import CartesianRepresentation, EarthLocation, SkyCoord

# We have defined a few helper functions in this `eclipse_helpers.py` file.
from eclipse_helpers import SOLAR_ECLIPSE_IMAGE, get_camera_metadata
from matplotlib.patches import Circle
from scipy import ndimage
from skimage.color import rgb2gray
from skimage.feature import peak_local_max
from skimage.transform import hough_circle, hough_circle_peaks
from sunpy.map.header_helper import make_fitswcs_header
from sunpy.net import Fido
from sunpy.net import attrs as a
from sunpy.time import parse_time

Read in Your Eclipse Photo

The first step is to read in our image. As mentioned above, we will be using an image taken during the 2017 eclipse taken with a consumer-grade camera. When reading in our image data, we’ll invert the y-axis and convert it to a grayscale image.

[2]:

im_rgb = np.flipud(matplotlib.image.imread(SOLAR_ECLIPSE_IMAGE))
im = rgb2gray(im_rgb)

/Users/nabil/micromamba/envs/sunpy-dev/lib/python3.13/site-packages/skimage/color/colorconv.py:984: RuntimeWarning: divide by zero encountered in matmul
  return rgb @ coeffs
/Users/nabil/micromamba/envs/sunpy-dev/lib/python3.13/site-packages/skimage/color/colorconv.py:984: RuntimeWarning: overflow encountered in matmul
  return rgb @ coeffs
/Users/nabil/micromamba/envs/sunpy-dev/lib/python3.13/site-packages/skimage/color/colorconv.py:984: RuntimeWarning: invalid value encountered in matmul
  return rgb @ coeffs

[3]:

plt.imshow(im, origin="lower", cmap="gray")
plt.show()

In order to be able to align our image with solar images from NASA, we will also need some additional metadata from our image. The two most important things we need to know are:

the GPS coordinates of where the photo was taken and
the time the image was taken

We can pull this metadata from the file and then use an additional function we wrote to process this metadata into a Python dictionary.

[4]:

with Path(SOLAR_ECLIPSE_IMAGE).open("rb") as f:
    tags = exifread.process_file(f)

[5]:

camera_metadata = get_camera_metadata(tags)

As it turns out, the time on the camera used to take this eclipse photo was wrong, we have to manually correct it.

[6]:

camera_metadata["time"] = parse_time("2017-08-21 17:27:13")

Find the Moon

Now that we’ve loaded our image and accompanying metadata, the next step is to locate the edge of the Moon in our image. This allows us to find the center of the Moon, which is needed when aligning our data, as well as allowing us to determine the scale of the Moon in the image. In order to do this we are going to use several different image manipulation techniques.

We start with applying a Gaussian blur to help segment the lunar disk from the corona and mask all parts of the image above a given threshold.

[7]:

blur_im = ndimage.gaussian_filter(im, 8)
mask = blur_im > blur_im.mean() * 3
plt.imshow(mask)
plt.show()

We then label those masked regions and select only the parts of the image that correspond to the bright, diffuse corona.

[8]:

label_im, nb_labels = ndimage.label(mask)
slice_y, slice_x = ndimage.find_objects(label_im == 1)[0]
roi = blur_im[slice_y, slice_x]

The next step is to detect the inner edge of the bright corona. To do this, we apply a Sobel filter in both the x and y directions, and then calculate a single image from the two directions.

[9]:

sx = ndimage.sobel(roi, axis=1, mode="constant")
sy = ndimage.sobel(roi, axis=0, mode="constant")
sob = np.hypot(sx, sy)

Finally, we use scikit-image to apply the Hough Transform to identify circles in the image. We then use this to extract the size in pixels of the lunar disk and its center.

[10]:

hough_radii = np.arange(np.floor(np.mean(sob.shape) / 4), np.ceil(np.mean(sob.shape) / 2), 10)
hough_res = hough_circle(sob > (sob.mean() * 5), hough_radii)

# Select the most prominent circle
accums, cx, cy, radii = hough_circle_peaks(hough_res, hough_radii, total_num_peaks=1)

As shown below, we now have a list of pixel coordinates corresponding to the solar limb in our image. The first frame is the cropped original image. The middle frame is the Sobel filtered image used to apply the Hough transform. The right frame is the fitted circle on the original image.

[11]:

fig, ax = plt.subplots(ncols=3, nrows=1, figsize=(9.5, 6))
ax[0].imshow(im[slice_y, slice_x])
ax[0].set_title("Original")
ax[1].imshow(sob > (sob.mean() * 5))
ax[1].set_title("Sobel")
circ = Circle(
    [cx, cy], radius=radii, facecolor="none", edgecolor="red", linewidth=2, linestyle="dashed", label="Hough fit"
)
ax[2].imshow(im[slice_y, slice_x])
ax[2].add_patch(circ)
ax[2].set_title("Original with fit")
plt.legend()
plt.show()

Lastly, let’s add units to our circle that we fit the lunar limb.

[12]:

im_cx = (cx[0] + slice_x.start) * u.pix
im_cy = (cy[0] + slice_y.start) * u.pix
im_radius = radii[0] * u.pix

Make a SunPy `Map`

At this point, we have our image data, it’s metadata and we’ve located the Moon. Now we are ready to load our eclipse photograph into a SunPy Map! A Map allows us to carry around both the data and metadata of our image together and, importantly, makes it easy to combine solar observations from multiple observatories.

First, we define the observer location (i.e., where on Earth did we take our picture?) and the orientation of the Sun in the sky. For the observer location, we can use the GPS coordinates from our image metadata.

[13]:

loc = EarthLocation(lat=camera_metadata["gps"][0], lon=camera_metadata["gps"][1], height=camera_metadata["gps"][2])
observer = loc.get_itrs(camera_metadata["time"])

Second, we determine the angular size of the Moon using its radius and its distance from the observer.

[14]:

moon = SkyCoord(sunpy.coordinates.get_body_heliographic_stonyhurst("moon", camera_metadata["time"], observer=observer))
R_moon = 0.2725076 * R_earth  # IAU mean radius
dist_moon = SkyCoord(observer).separation_3d(moon)
moon_obs = np.arcsin(R_moon / dist_moon).to("arcsec")
print(moon_obs)

2025-06-23 09:08:55 - sunpy - INFO: Apparent body location accounts for 1.23 seconds of light travel time

INFO: Apparent body location accounts for 1.23 seconds of light travel time [sunpy.coordinates.ephemeris]
975.9073137731282 arcsec

Combining this angular radius with the radius of the lunar disk in pixels gives us the angular size of the pixels in the image. In the parlance of astronomical imaging, this is often referred to as the plate scale.

[15]:

plate_scale = moon_obs / im_radius
print(plate_scale)

4.356729079344322 arcsec / pix

We also use the observer location to calculate the orientation of the Sun as seen from that location on Earth. This gives us a rotation angle between our image coordinate system and the solar north pole. If your camera is not perfectly horizontal then you may need to fudge this value slightly.

[16]:

solar_rotation_angle = sunpy.coordinates.sun.orientation(loc, camera_metadata["time"])
print(solar_rotation_angle)

-54d19m44.13467961s

Finally we have all the information we need to build a sunpy Map for our eclipse image.

[17]:

frame = sunpy.coordinates.Helioprojective(observer=observer, obstime=camera_metadata["time"])
moon_hpc = moon.transform_to(frame)

header = make_fitswcs_header(
    im,
    moon_hpc,
    reference_pixel=u.Quantity([im_cx, im_cy]),
    scale=u.Quantity([plate_scale, plate_scale]),
    rotation_angle=solar_rotation_angle,
    exposure=camera_metadata.get("exposure_time"),
    instrument=camera_metadata.get("camera_model"),
    observatory=camera_metadata.get("author"),
)

[18]:

eclipse_map = sunpy.map.Map(im, header)

[19]:

fig = plt.figure(figsize=(9, 9))
ax = plt.subplot(projection=eclipse_map)
eclipse_map.plot(axes=ax)
plt.show()

Find a Star

Though we already have all of the metadata we need to make a SunPy Map, we can further improve the accuracy by locating a known star in the image and using that to determine the rotation angle. In the case of the 2017 eclipse the very bright star (magnitude 1.4) Regulus was close to the Sun and in the field of view of our photograph. For the 2024 eclipse, no such bright star will be visible, which may make this method of aligning your image challenging. The best candidate looks to be Alpha Piscium which is a binary system with a combined magnitude of 3.82, significantly dimmer than Regulus. You can see the stars close to the Sun by using Stellarium.

As Regulus is a well known star, we can create a coordinate object for it, including its distance.

[20]:

regulus = SkyCoord(ra="10h08m22.311s", dec="11d58m01.95s", distance=79.3 * u.lightyear, frame="icrs")
print(regulus)

<SkyCoord (ICRS): (ra, dec, distance) in (deg, deg, lyr)
    (152.0929625, 11.96720833, 79.3)>

We can then plot the expected location of Regulus on our eclipse image. We set the scaling such that it make Regulus more visible and zoom in on the relevant part of the field of view. You can see that the expected location and the actual location are quite different.

[21]:

fig = plt.figure(figsize=(9, 9))
ax = plt.subplot(projection=eclipse_map)
eclipse_map.plot(axes=ax, clip_interval=(0, 90) * u.percent)
ax.plot_coord(
    regulus, "o", markeredgewidth=0.5, markeredgecolor="w", markerfacecolor="None", markersize=10, label="Regulus"
)
plt.legend()
plt.xlim(100, 500)
plt.ylim(0, 500)
plt.show()

Given this offset, we want to fix our image metadata such that the actual and expected locations of Regulus are line up. We can calculate the expected distance from the center of the Sun to Regulus in pixels.

[22]:

regulus_pixel = CartesianRepresentation(*eclipse_map.wcs.world_to_pixel(regulus), 0) * u.pix
sun_pixel = (
    CartesianRepresentation(*eclipse_map.wcs.world_to_pixel(SkyCoord(0 * u.arcsec, 0 * u.arcsec, frame=frame)), 0)
    * u.pix
)
regulus_r = (regulus_pixel - sun_pixel).norm()
print(regulus_r)

1084.0811009031981 pix

We then use this to find Regulus in our image, by filtering out all pixels which are closer to the Sun than this.

[23]:

pix_x = np.arange(eclipse_map.dimensions[0].value) * u.pix - sun_pixel.x
pix_y = np.arange(eclipse_map.dimensions[1].value) * u.pix - sun_pixel.y
xx, yy = np.meshgrid(pix_x, pix_y)
r = np.sqrt(xx**2 + yy**2)

filter_r = regulus_r - (regulus_r / 5)

masked = im.copy()
masked[r < filter_r] = masked.min()

Having masked out most of the Sun and its corona, we now find the highest peak remaining, which should be Regulus.

[24]:

regulus_y, regulus_x = peak_local_max(masked, min_distance=2, num_peaks=1)[0]
actual_regulus_pixel = CartesianRepresentation(regulus_x, regulus_y, 0) * u.pix

We can now compare the identified pixel coordinates of Regulus to the expected coordinates assuming the camera was exactly horizontal.

[25]:

print(actual_regulus_pixel)
print(regulus_pixel)

(372., 247., 0.) pix
(339.45349652, 307.65804366, 0.) pix

Finally, we calculate the angular offset between our expected location and our identified location and then add this difference to correct our solar rotation angle.

[26]:

vec_expected = regulus_pixel - sun_pixel
vec_actual = actual_regulus_pixel - sun_pixel
fudge_angle = -np.arccos(vec_expected.dot(vec_actual) / (vec_expected.norm() * vec_actual.norm()))
print(fudge_angle.to(u.deg))

-3.5738144694153595 deg

We then use this correction factor to build a new Map for our image with updated metadata.

[27]:

header = make_fitswcs_header(
    im,
    moon_hpc,
    reference_pixel=u.Quantity([im_cx, im_cy]),
    scale=u.Quantity([plate_scale, plate_scale]),
    rotation_angle=solar_rotation_angle + fudge_angle,
    exposure=camera_metadata["exposure_time"],
    instrument=camera_metadata["camera_model"],
)

[28]:

eclipse_map = sunpy.map.Map(im, header)

Plotting our image again, we now find that the identified location of Regulus and our image line up much better.

[29]:

fig = plt.figure(figsize=(9, 9))
ax = plt.subplot(projection=eclipse_map)
eclipse_map.plot(axes=ax, clip_interval=(0, 90) * u.percent)
ax.plot_coord(
    regulus, "o", markeredgewidth=0.5, markeredgecolor="w", markerfacecolor="None", markersize=10, label="Regulus"
)
plt.legend()
plt.xlim(100, 500)
plt.ylim(0, 500)
plt.show()

Combine your Image with NASA Data

As mentioned above, one of the main advantages of having data in a Map is that it is then easy to combine observations from multiple different sources. Let’s overlay an AIA image from the SDO satellite. We’ll choose an image that shows extreme ultraviolet emission from the corona, revealing plasma that is around one million degrees. Fortunately, all SDO data is publicly available and SunPy provides a convenient interface for searching for and downloading this data.

[30]:

aia_result = Fido.search(
    a.Time("2017-08-21 17:27:00", "2017-08-21 17:45:00", eclipse_map.date),
    a.Instrument("AIA"),
    a.Wavelength(171 * u.Angstrom),
)
print(aia_result)

Results from 1 Provider:

1 Results from the VSOClient:
Source: https://sdac.virtualsolar.org/cgi/search
Data retrieval status: https://docs.virtualsolar.org/wiki/VSOHealthReport
Total estimated size: 67.789 Mbyte

       Start Time               End Time        Source Instrument   Wavelength   Provider  Physobs  Wavetype Extent Width Extent Length Extent Type   Size
                                                                     Angstrom                                                                        Mibyte
----------------------- ----------------------- ------ ---------- -------------- -------- --------- -------- ------------ ------------- ----------- --------
2017-08-21 17:27:09.000 2017-08-21 17:27:10.000    SDO        AIA 171.0 .. 171.0     JSOC intensity   NARROW         4096          4096    FULLDISK 64.64844

[31]:

aia_result[0, 0]

[31]:

QueryResponseRow index=0

Start Time	End Time	Source	Instrument	Wavelength	Provider	Physobs	Wavetype	Extent Width	Extent Length	Extent Type	Size	fileid
				Angstrom							Mibyte
Time	Time	str3	str3	float64[2]	str4	str9	str6	str4	str4	str8	float64	str24
2017-08-21 17:27:09.000	2017-08-21 17:27:10.000	SDO	AIA	171.0 .. 171.0	JSOC	intensity	NARROW	4096	4096	FULLDISK	64.64844	aia__lev1:171:1282411667

[32]:

files = Fido.fetch(aia_result[0, 0], site="NSO")

[33]:

files

[33]:

<parfive.results.Results object at 0x158950690>
['/Users/nabil/sunpy/data/aia.lev1.171A_2017_08_21T17_27_09.35Z.image_lev1.fits']

Having downloaded this data we create a SunPy Map and then plot it on top of our eclipse image. Note that despite not being in the same coordinate system, our AIA Map is automatically transformed to the coordinate system of our image before plotting.

[34]:

aia_map = sunpy.map.Map(files[0])

[35]:

fig = plt.figure(figsize=(9, 9))
ax = plt.subplot(projection=eclipse_map)
eclipse_map.plot(axes=ax)
aia_map.plot(axes=ax, autoalign=True)
aia_map.draw_grid(axes=ax)
plt.show()

2025-06-23 09:09:07 - sunpy - INFO: Using mesh-based autoalignment

INFO: Using mesh-based autoalignment [sunpy.map.mapbase]

Conclusion

In this blog post, we demonstrated how to read your eclipse images into a SunPy Map and how to combine your own photographs with data from NASA observations of the Sun. Though this post used data from the 2017 eclipse, you should be able to use the same techniques to process your 2024 eclipse observations. Happy eclipse viewing!

pyOpenSci and sunpy

2024-01-24T00:00:00+00:00

pyOpenSci is a “diverse community of people interested in building a community of practice around scientific software written in Python”. They are an exciting community working on improving the information around the packaging and tooling used by the scientific Python community as well to provide support for package maintainers. There is a review process to become a pyOpenSci accepted package which comes with two reviews from independent members of the wider community.

With this in mind, the SunPy Project decided to submit the sunpy package to pyOpenSci to ensure that it is aligned with the community that pyOpenSci is building, and increase visibility of SunPy.

The review was started on the pyOpenSci software-submission repository. The entire review process is open, with one editor and two reviewers and the entire exchange between the sunpy maintainers and the reviewers are on that issue. This process was incredibly helpful as it highlighted areas of our documentation which could be improved and some technical choices that were made several years ago but now were redundant or not best practice.

After these issues were fixed, sunpy was accepted into pyOpenSci and we added the badge to the sunpy readme. The plan in future is to submit more SunPy Project packages as well as integrate more closely with pyOpenSci to replace our own affiliated package system. Our hope is that by aligning with pyOpenSci we can help to contribute back to the wider community.

The future of time series data in sunpy

2023-05-10T00:00:00+00:00

In late 2022 I got a small development grant from NumFocus to scope the future of time series data in sunpy. The successful application can be read on the sunpy wiki. The application contains context that I won’t repeat here. This blog post is the key outcome of this grant, with a record of what I did, the recommendations I made, and any decisions we came to as a community.

User requirements

The first stage of my work investigated what the user requirements are for a sunpy data container for time series data. As part of this I used my own experience and the following community engagement:

Discussion at one of the weekly sunpy community meetings in December 2022
Discussion on the Python in Heliophysics mailing list
Discussion on the SunPy forum

From these discsusions came the following list of requirements:

Requirement	Notes
Store data that is a function of time	This means the time column should be treated as the index or coordinates to the data, and be stored as a time-like type.
Handle different time scales	Data can have times defined in a variety of different time scales (e.g. UTC, TAI)
Store multi-dimensional data	Although time is a common index to timeseries data, it isn’t always the only one. As an example, velocity distribution functions measured in the solar wind are 4D datasets, with data as a function of time and three dimensions in velocity space.
Handle time scales with leapseconds	Some timescales can contain timestamps that occur within a leapsecond.
Store and use physical units with the data and any non-time indices
Store data in a format that can be used with scientific Python libraries
Support for storing out-of memory datasets
Store metadata alongside actual data
Have a way to store an observer coordinate alongside the time index
Have an easy way to do common data manipulation tasks	e.g. interpolating, resampling, rebinning
Have a way to combine multiple timeseries objects, and keep track of metadata
Ability to convert to other common time series objects (e.g. `pandas.DataFrame`)
Functionality for loading and saving out to common file formats

Existing options for a data container

The next step was to identify a set of possible data containers that could be used to store time- series data in sunpy. The identified options were:

astropy.timeseries.TimeSeries
pandas.DataFrame
xarray.DataArray (or xarray.DataSet)
numpy.ndarray
ndcube

What do other projects use?

I also looked at what Python in Heliophysics projects use (as of writing, in Jan 2023):

Package	Container
sunpy	Custom `TimeSeries` object, backed by `pandas.DataFrame`
HAPI Client	`numpy.ndarray`
pySPEDAS	Not clear if users can access the data itself
spacepy	Unclear if there is any specific timeseries container object
aidapy	`xarray.DataArray`
cdflib	`numpy.ndarray`
NDCube	`NDCube`
pytplot	`xarray.DataArray`
solo-epd-loader	`pandas.DataFrame`
speasy	Custom `DataContainer` object, backed by `numpy.ndarray`

There is no common container used, with only astropy.TimeSeries not represented out of the possible options above.

What datasets does sunpy currently support?

sunpy currently has built in support for reading CDF files that conform to the Space Physics Guidelines for CDF, as long as the dataset is one- or two- dimensional. Alongside this several custom data readers have been written to support different data sources:

(links point to the data source information web page)

Data product(s)	File format
SDO EVE/ESP L1	FITS
SDO EVE/ESP L0CS	Text file
FERMI GBM summary	FITS
GOES XRS	FITS, netCDF
PROBA-2 LYRA ligthcurve	FITS
NOAA solar cycle monthly indices	JSON
NOAA solar cycle predicted indices	JSON
NoRH radio	FITS
RHESSI x-ray summary	FITS

Evaluating options

Having found possible options, in this section I’ve evaluated them against the criteria set out above.

`numpy.ndarray`


Time-like index data	🛑	Can store datetime64 data, but no support for indexes
Different time scales	🛑	No support
Multi-dimensional data	🟩
Physical units	🛑	No support
Interop with scientific Python	🟩
Out of memory	🛑	numpy arrays are always in memory
Metadata	🛑	No support
Observer coordinates	🛑	No support
Easy data manipulation	🟩
I/O	🟠	Can save to binary .npy format or text file

`pandas.DataFrame`


Time-like index data	🟩
Different time scales	🛑	No support
Multi-dimensional data	🟠	Possible, but recommended to use xarray instead
Physical units	🛑	No native support (tracking issue), could be possible with `pint-pands`
Interop with scientific Python	🟩
Out of memory	🛑	pandas DataFrames are always in memory
Metadata	🟩	Possible to add additional properties to a DataFrame
Observer coordinates	🛑	No support
Easy data manipulation	🟩	Many built in methods for manipulating time-like data
I/O	🟩	Lots of I/O options

`xarray.DataArray`


Time-like index data	🟩
Different time scales	🛑	No support
Multi-dimensional data	🟩
Physical units	🛑	No native support (tracking issue), could be possible with `pint-xarray`
Interop with scientific Python	🟩
Out of memory	🟩	Support for computing using `dask`
Metadata	🟩	Possible to add metadata to a DataArray
Observer coordinates	🟠	Support for adding “non-dimensional” coordinates (e.g. longitude/latitude), but not clear if storing astropy SkyCoord would work
Easy data manipulation	🟩	Many built in methods for manipulating time-like data
I/O	🟩	Lots of I/O options

`astropy.timeseries.TimeSeries`


Time-like index data	🟩
Different time scales	🟩
Multi-dimensional data	🛑
Physical units	🟩
Interop with scientific Python	🟩
Out of memory	🟠	Apparently there is some support, but this is undocumented.
Metadata	🟩	Can store on the `.meta` attribute
Observer coordinates	🟩
Easy data manipulation	🟠
I/O	🟩	I/O is done via `astropy.table.Table`

`NDCube`


Time-like index data	🟩
Different time scales	🟩
Multi-dimensional data	🟩
Physical units	🟩
Interop with scientific Python	🟩
Out of memory	🟠	Seems to be supported in theory, but little docs
Metadata	🟩	Can store arbitrary FITS metadata
Observer coordinates	🛑	No support for extra coordinates
Easy data manipulation	🛑	Very few manipulation methods implemented
I/O	🛑

Initial recommendations

numpy.ndarray doesn’t implement several key features, and these are almost certainly out of scope for future ndarray development, so I suggest ndarray is discounted.
xarray.DataArray builds on top of pandas.DataFrame with additional features that would be useful to us, I suggest pandas.DataFrame is discounted.
NDCube is designed specifically to store data that is associated with a FITS world coordinate system (WCS). While some solar timeseries data is already in the FITS format, a large portion is in CDF format which is tabular, which FITS is not primarily designed to represent. So I suggest NDCube is discounted.

At a SunPy community meeting there was a consensus agreement that going forward we should consider astropy.TimeSeries and xarray.DataArray as the two options to consider.

These two options have the following comparison:

	`astropy.TimeSeries`	`xarray.DataArray`
Time-like index data	🟩	🟩
Different time scales	🟩	🛑
Multi-dimensional data	🛑	🟩
Physical units	🟩	🛑
Interop with scientific Python	🟩	🟩
Out of memory	🟠	🟩
Metadata	🟩	🟩
Observer coordinates	🟩	🟠
Easy data manipulation	🟠	🟩
I/O	🟩	🟩

My initial recommendation would be to adopt xarray.DataArray, as the two red items have a strong possibility of being solved with DataArray:

It should (I haven’t confirmed this) be possible to convert times in different time scales (including ones with leap seconds) to a single timescale that doesn’t have leap seconds, and store this in an xarray.DataArray.
Although there is not native support for units in DataArray currently, there is interest and ongoing development to support them.

It is unclear to me (because I did not have time to investigate) how hard it would be to implement support for storing rich coordinates (ie. astropy.SkyCoord) in the extra_coords part of xarray data structures.

In contrast I think implementing multi-dimensional data in astropy.TimeSeries, adding documentation for out of memory datasets, and implementing easy data manipulation methods would take significantly more effor than this. Finally, xarray has a much bigger development community than astropy.TimeSeries, so implementing bug fixes and new features would probably be much easier with xarray.

Putting `astropy` objects in `xarray` structures

For the final part of the small development grant, I investigated the changes needed to put astropy objects in xarray structures.

As a model for doing this, it is currently possible to store unitful data created with pint in xarray structures. Support for doing this has two components:

xarray natively supports storing duck arrays
xarray-pint provides a set of accessors that can be used to serialise and deserialise unitful data so that it can be saved to a file and loaded again. It does this by converting the unit data into metadata, with strings representing units.

It is not currently possible to store astropy.Quantity objects in xarray structures, as they inherit directly from ndarray, and get coerced from Quantity to ndarray during the xarray structure initialisation. I think fixing this is (at least initially) a one line change, changing (what was xarray/core/variable.py#L288 on commit hash 51554f2638bc9e4a527492136fe6f54584ffa75d) from

data = np.asarray(data)

if not isinstance(data, np.array):
    data = np.asarray(data)

Before moving forward with this it needs to be possible to run the full unit tests in xarray with astropy.Quantity. I started work on this in these two PRs:

aiapy: A SunPy affiliated package for analyzing data from the Atmospheric Imaging Assembly

2022-01-06T00:00:00+00:00

This blog post was written in a Jupyter notebook. Click here for an interactive version:

[1]:

import aiapy
import astropy
import astropy.time
import astropy.units as u
import matplotlib as mpl
import matplotlib.pyplot as plt
import numpy as np
import sunpy
import sunpy.map
from aiapy.calibrate import (
    correct_degradation,
    degradation,
    estimate_error,
    fetch_spikes,
    register,
    respike,
    update_pointing,
)
from aiapy.calibrate.util import get_correction_table, get_error_table, get_pointing_table
from aiapy.psf import deconvolve, psf
from aiapy.response import Channel
from astropy.coordinates import SkyCoord
from astropy.visualization import ImageNormalize, LogStretch, time_support
from sunpy.net import Fido
from sunpy.net import attrs as a
from sunpy.time import parse_time

# Increases the figure size in this notebook.
mpl.rcParams["savefig.dpi"] = 150
mpl.rcParams["figure.dpi"] = 150

The Atmospheric Imaging Assembly (AIA) instrument onboard the NASA Solar Dynamics Observatory (SDO) spacecraft has observed the full- disk of the Sun, nearly continuously, for the last ten years. It is one of three instruments on SDO, along with the Helioseismic and Magnetic Imager (HMI) and the Extreme Ultraviolet Variability Experiment (EVE). AIA is a narrowband imaging instrument comprised of four separate telescopes that collectively observe the full-disk of the Sun at ten different wavelengths: seven extreme ultraviolet (EUV) wavelengths, two far UV wavelengths, and one visible wavelength.

aiapy is a SunPy-affiliated Python package for analyzing calibrated (level 1) imaging data from AIA. It includes capabilities for aligning images between channels, deconvolving images with the instrument point-spread function (PSF), computing channel sensitivity as a function of wavelength, and correcting images for telescope degradation, among others. These capabilities are divided between the three subpackages of aiapy:

aiapy.calibrate
aiapy.psf
aiapy.response

In this blog post, we will demonstrate the functions in each of these subpackages on a set of example level 1 AIA images.

Before we begin, we make a note of what versions of astropy, sunpy, and aiapy were used in the writing of this blog post.

[2]:

print(f"astropy v{astropy.__version__}")
print(f"sunpy v{sunpy.__version__}")
print(f"aiapy v{aiapy.__version__}")

astropy v7.0.1
sunpy v6.1.1
aiapy v0.10.1

Obtaining AIA Data

First, we will use the Fido client to fetch two AIA images from the VSO: one from the 171 Å channel and one from the 211 Å channel.

[3]:

t_start = parse_time("2017-09-10T20:00:00")
search_results = Fido.search(
    a.Time(t_start, t_start + 12 * u.s),
    a.Instrument.aia,
    a.Wavelength(171 * u.angstrom) | a.Wavelength(211 * u.angstrom),
)
search_results

[3]:

Results from 2 Providers:

1 Results from the VSOClient:

VSOQueryResponseTable length=1

Start Time	End Time	Source	Instrument	Wavelength	Provider	Physobs	Wavetype	Extent Width	Extent Length	Extent Type	Size
				Angstrom							Mibyte
Time	Time	str3	str3	float64[2]	str4	str9	str6	str4	str4	str8	float64
2017-09-10 20:00:09.000	2017-09-10 20:00:10.000	SDO	AIA	171.0 .. 171.0	JSOC	intensity	NARROW	4096	4096	FULLDISK	64.64844

1 Results from the VSOClient:

VSOQueryResponseTable length=1

Start Time	End Time	Source	Instrument	Wavelength	Provider	Physobs	Wavetype	Extent Width	Extent Length	Extent Type	Size
				Angstrom							Mibyte
Time	Time	str3	str3	float64[2]	str4	str9	str6	str4	str4	str8	float64
2017-09-10 20:00:12.000	2017-09-10 20:00:13.000	SDO	AIA	211.0 .. 211.0	JSOC	intensity	NARROW	4096	4096	FULLDISK	64.64844

[4]:

files = Fido.fetch(search_results)

We can then create sunpy.map.Map objects from the downloaded files.

[5]:

m_171, m_211 = sunpy.map.Map(sorted(files))

[6]:

m_171.peek(vmin=0)
m_211.peek(vmin=0)

Note that both the sunpy and aiapy documentation pages include more detailed examples of how to obtain AIA data from the JSOC as well

More complex queries (e.g. searching by exposure time or quality keyword) can be constructed by searching the JSOC directly via the drms package. See the last example listed above for an example of this.

PSF Deconvolution

AIA images are subject to convolution with the instrument PSF due to effects introduced by the filter mesh of the telescope and the CCD, among others. This has the effect of “blurring” the image. The PSF diffraction pattern may also be particularly noticeable during the impulsive phase of a flare where the intensity enhancement is very localized. To remove these artifacts, the PSF must be deconvolved from the level 1 image. This is because the PSF itself is defined on the pixel grid of the CCD.

We’ll calculate the PSF for the 171 Å channel using the psf function in the aiapy.psf subpackage. The PSF model accounts for several different effects, including diffraction from the mesh grating of the filters, charge spreading, and jitter. This implementation of the PSF calculation follows Grigis et al (2012). Further details on how the PSF is calculated can be found in that document. Currently, this only works for \(4096\times4096\) full frame images.

Note: The following cell will be significantly slower (many minutes) if you do not have a NVIDIA GPU and the cupy package installed. The GPU-enabled calculation has been benchmarked at ~5 s on an NVIDIA RTX 4090. Use of the GPU can be explicitly disabled by setting use_gpu=False.

[7]:

psf_171 = psf(m_171.wavelength)

We can then visualize the PSF. The diffraction “arms” extending from the center pixel can often be seen in flare observations due to the intense, small-scale brightening.

[8]:

plt.imshow(psf_171, origin="lower", norm=ImageNormalize(vmax=1e-6, stretch=LogStretch()))
plt.colorbar()

[8]:

<matplotlib.colorbar.Colorbar at 0x70adf80b3ed0>

We can then use the calculated PSF to deconvolve our 171 Å image using the deconvolve function in aiapy.psf. This deconvolution algorithm is an implementation of the well-known Richardson-Lucy algorithm. If the PSF is not explicitly passed in, the PSF for that wavelength will be computed automatically. When deconvolving many images, it is highly recommend to precompute the PSF function for each desired wavelength.

NOTE: The deconvolution of the image can also be optionally accelerated with a NVIDIA GPU and cupy. On a Titan RTX, deconvolution of a full \(4096\times4096\) image has been benchmarked at \(\le1\) s. On a CPU, the equivalent operation should take ~25 s.

[9]:

m_171_deconvolved = deconvolve(m_171, psf=psf_171)

2025-03-03 14:31:50 - sunpy - INFO: Using a GPU via cupy

INFO: Using a GPU via cupy [aiapy.psf.deconvolve]

We can now zoom in on some loop structures that extend off the limb to examine the differences between our image before and after deconvolution.

[10]:

blc = SkyCoord(750, -375, unit="arcsec", frame=m_171.coordinate_frame)
fov = {"width": 400 * u.arcsec, "height": 400 * u.arcsec}
m_171_cutout = m_171.submap(blc, **fov)
m_171_deconvolved_cutout = m_171_deconvolved.submap(blc, **fov)

[11]:

fig = plt.figure(figsize=(7, 3))
ax = fig.add_subplot(121, projection=m_171_cutout)
m_171_cutout.plot(axes=ax, title="Before Deconvolution")
ax = fig.add_subplot(122, projection=m_171_deconvolved_cutout)
m_171_deconvolved_cutout.plot(axes=ax, title="After Deconvolution")
ax.coords[1].set_axislabel(" ")

Note that comparing the images on the left and right, deconvolving with the PSF removes the appearance of the diffraction pattern near the bright looptop. This also brings out more detail in the strands of the post-flare arcade.

We can see this more clearly by plotting the intensity of the original and deconvolved images across the top of the looptop.

[12]:

x = np.linspace(m_171_deconvolved_cutout.dimensions.x.value * 0.55, m_171_deconvolved_cutout.dimensions.x.value * 0.7)
y = 0.59 * m_171_deconvolved_cutout.dimensions.y.value * np.ones(x.shape)
sl = np.s_[np.round(y[0]).astype(int), np.round(x[0]).astype(int) : np.round(x[-1]).astype(int)]

[13]:

fig = plt.figure(figsize=(7, 3))
ax = fig.add_subplot(121, projection=m_171_deconvolved_cutout)
m_171_deconvolved_cutout.plot(axes=ax)
ax.plot(x, y, lw=1)
ax = fig.add_subplot(122)
Tx = sunpy.map.all_coordinates_from_map(m_171_cutout)[sl].Tx
ax.plot(Tx, m_171_cutout.data[sl], label="Original")
ax.plot(Tx, m_171_deconvolved_cutout.data[sl], label="Deconvolved")
ax.set_ylabel(f"Intensity [{m_171_cutout.unit}]")
ax.set_xlabel(r"Helioprojective Longitude [arcsec]")
ax.legend(loc="upper center", ncol=2, frameon=False, bbox_to_anchor=(0.5, 1.15))
plt.tight_layout()

Respiking Level 1 Images

The aiapy.calibrate module contains a number of useful functions, including the ability to “respike” images through the respike function. AIA level 1 images have been corrected for hot-pixels (commonly referred to as “spikes”) using an automated correction algorithm which detects them, removes them, and replaces the “holes” left in the image via interpolation. However, for certain research topics, this automated hot-pixel removal process may result in unwanted removal of bright points which may be physically meaningful. The respike function allows you to revert this removal by putting back all the removed pixel values. This corresponds to the aia_respike.pro IDL procedure as described in the SDO Analysis Guide.

We will create a “respiked” version of our level 1 171 Å image. This function will query the JSOC for the relevant “spikes” file which contains the locations (in pixel coordinates) and associate intensities for each of the hot pixels. The hot pixel values are then replaced in the image at the appropriate locations.

[14]:

m_171_respiked = respike(m_171)

[15]:

fig = plt.figure(figsize=(7, 3))
ax = fig.add_subplot(121, projection=m_171)
m_171.plot(axes=ax)
ax.set_title(f"Despiked (Level {m_171.processing_level:.0f})")
ax = fig.add_subplot(122, projection=m_171_respiked)
m_171_respiked.plot(axes=ax)
ax.set_title(f"Respiked (Level {m_171_respiked.processing_level})")
ax.coords[1].set_axislabel(" ")

Looking at the two images, it is not obvious by visual inspection what the differences are between the despiked and respiked maps. We can use the fetch_spikes functions (which is called automatically by respike) to find the intensity values and the associated pixel coordinates

[16]:

pix_coords, vals = fetch_spikes(m_171)

We can then use the pixel coordinates to find the “despiked” intensity values and plot the distributions of the “despiked” and “respiked” values.

[17]:

vals_despiked = m_171.data[pix_coords.y.value.round().astype(int), pix_coords.x.value.round().astype(int)]

[18]:

plt.hist(vals, bins="scott", log=True, histtype="step", label="Respiked")
plt.hist(vals_despiked, bins="scott", log=True, histtype="step", label="Despiked")
plt.legend()
plt.xlabel(f"Intensity [{m_171.unit.to_string()}]")
plt.ylabel("Number of Pixels")

[18]:

Text(0, 0.5, 'Number of Pixels')

Additionally, we can plot the coordinates of each of our hot pixels on our “respiked” map. We can convert the above pixel coordinates to world coordinates using the .pixel_to_world() method on our level 1 map. Note that we can also return the world coordinates from the fetch_spikes function automatically by passing in the as_coords=True keyword argument.

[19]:

spike_coords = m_171.pixel_to_world(pix_coords.x, pix_coords.y)

[20]:

fig = plt.figure()
ax = fig.add_subplot(111, projection=m_171_respiked)
m_171_respiked.plot(axes=ax)
ax.plot_coord(spike_coords, marker=".", ls=" ", markersize=1)

[20]:

[<matplotlib.lines.Line2D at 0x70ad8c28a0d0>]

For a more detailed example of respiking level 1 images, see the respiking example in the aiapy example gallery.

Transforming Level 1 Images to Level 1.5

One of the most common operations performed on level 1 AIA images is “aiaprep,” commonly performed with the aia_prep.pro procedure in IDL. This prep process, which promotes level 1 images to level 1.5, is a combination of two steps:

update pointing and
image registration.

The first step is accomplished through the update_pointing function in aiapy.calibrate. This function queries the 3-hourly master pointing table stored at the JSOC to update relevant FITS keywords.

[21]:

table = get_pointing_table("JSOC", time_range=[m_171.date - 1 * u.day, m_171.date + 1 * u.day])
m_171_up = update_pointing(m_171, pointing_table=table)

We can verify this step by examining the reference pixel before and after the pointing update.

[22]:

m_171.reference_pixel

[22]:

PixelPair(x=<Quantity 2052.300049 pix>, y=<Quantity 2048.280029 pix>)

[23]:

m_171_up.reference_pixel

[23]:

PixelPair(x=<Quantity 2052.33252 pix>, y=<Quantity 2048.275391 pix>)

The second step is accomplished through register function in aiapy.calibrate. This function performs an affine transform on the level 1 pixel grid to remove the roll angle, scale the image to resolution of 0.6 arcseconds-per-pixel, and shift the center of the image to the center of the Sun. The original intensity values are then reinterpolated onto this transformed grid.

[24]:

m_171_l15 = register(m_171_up)

We can verify that the image has been transformed appropriately by comparing at the scale and rotation_matrix attributes before and after registration. Note that the rotation_matrix is now diagonalized, meaning that the world and pixel grids are aligned.

[25]:

print(m_171_up.scale)
print(m_171_up.rotation_matrix)

SpatialPair(axis1=<Quantity 0.599489 arcsec / pix>, axis2=<Quantity 0.599489 arcsec / pix>)
[[ 9.99999944e-01 -3.35522089e-04]
 [ 3.35522089e-04  9.99999944e-01]]

[26]:

print(m_171_l15.scale)
print(m_171_l15.rotation_matrix)

SpatialPair(axis1=<Quantity 0.6 arcsec / pix>, axis2=<Quantity 0.6 arcsec / pix>)
[[ 1.00000000e+00 -5.90601768e-20]
 [-2.18097419e-20  1.00000000e+00]]

Note that we can also combine these two operations into a “prep” function to imitate aiaprep.pro.

[27]:

def prep(smap):
    table = get_pointing_table("JSOC", time_range=[smap.date - 1 * u.day, smap.date + 1 * u.day])
    return register(update_pointing(smap, pointing_table=table))

[28]:

m_211_l15 = prep(m_211)

Applying this “prep” procedure means that our 171 Å and 211 Å images are now interpolated to the same pixel grid.

[29]:

print(m_211_l15.scale)
print(m_211_l15.rotation_matrix)

SpatialPair(axis1=<Quantity 0.6 arcsec / pix>, axis2=<Quantity 0.6 arcsec / pix>)
[[1.00000000e+00 5.18338069e-20]
 [5.18338069e-20 1.00000000e+00]]

Before performing any operations between two maps from different channels, one should always register each image such that the pixel grids of each image are aligned. For more information on transforming level 1 images to level 1.5, see the following example in the aiapy example gallery:

Registering and aligning level 1 data

Degradation Correction

The performance of the AIA telescope is unfortunately degrading over time, leading to the resulting images becoming increasingly dim. We can correct for this by modeling the degradation over time using the degradation function. This function calculates the degradation factor \(d(t)\) as,

\[d(t) = \frac{A_{eff}(t_{e})}{A_{eff}(t_0)}(1 + p_1\delta t + p_2\delta t^2 + p_3\delta t^3),\]

where the leading coefficient is the ratio between the effective area at some epoch time \(t_e\) and the initial time \(t_0\). The term in the parentheses represents an interpolation from \(t_e\) to the selected time \(t\), where \(\delta t=t-t_e\). \(p_1,p_2,p_3\) as well as \(A_{eff}(t_e)\) and \(A_{eff}(t_0)\) are retrieved from the JSOC (by default) or a user-supplied table.

We can use this function to calculate the degradation of the 211 Å channel from the start of the SDO mission to now.

[30]:

t_begin = parse_time("2010-03-25T00:00:00")
now = astropy.time.Time.now()
time_window = t_begin + np.arange(0, (now - t_begin).to(u.day).value, 7) * u.day

First, we’ll query the degradation calibration table from the JSOC. Doing this ahead of time avoids repeated network calls when we want to repeatedly calculate the degradation.

[31]:

correction_table = get_correction_table("JSOC")

We can then calculate the 211 Å degradation using the above correction table.

[32]:

d_211 = degradation(m_211.wavelength, time_window, correction_table=correction_table)

Note that we can also use the .date attribute on our 211 Å map to compute the degradation at the time of that map

[33]:

d_211_map = degradation(m_211.wavelength, m_211.date, correction_table=correction_table)

The degradation calibration is routinely updated (e.g. using data from sounding rocket flights), with new versions added to the table returned from the JSOC. We can look at older versions of the calibration to see how the predicted degradation has changed using the calibration_version keyword argument. Note that the default version in aiapy v0.10.0 is 10.

And compare the two degradation curves as a function of time over the lifetime of the mission.

[34]:

with time_support(format="jyear"):
    plt.plot(time_window, d_211, label="v10")
    plt.plot(m_211.date, d_211_map, linestyle="", marker="o", color="C0", label=m_211.date.iso)
plt.ylabel("Degradation 211 Å")
plt.legend()

[34]:

<matplotlib.legend.Legend at 0x70ad7e10a210>

Additionally, aiapy.calibrate also includes the correct_degradation function which accepts a sunpy.map.Map, calculates the degradation factor, divides the image data by this degradation factor, and returns a new corrected map.

[35]:

m_211_corrected = correct_degradation(m_211, correction_table=correction_table)

[36]:

fig = plt.figure(figsize=(7, 3))
ax = fig.add_subplot(121, projection=m_211)
m_211.plot(axes=ax, vmin=0, vmax=2.5e3, title="Uncorrected")
ax = fig.add_subplot(122, projection=m_211_corrected)
m_211_corrected.plot(axes=ax, vmin=0, vmax=2.5e3, title="Corrected")
ax.coords[1].set_axislabel(" ")

Computing Uncertainties

One of the newest additions to aiapy is the function for computing uncertainties on AIA intensities. This is done through the estimate_error function in aiapy.calibrate. estimate_error is an exact port of the SSWIDL function aia_bp_estimate_error.pro. This calculation includes errors from the shot noise, read noise, quantization, and onboard compression. The calculation can also optionally include contributions from the photometric calibration and errors in the atomic data.

We can pass in the (unitful) data array attached to our 171 Å map and compute the intensities.

[37]:

error_table = get_error_table("SSW")
errors_171 = estimate_error(m_171_l15.quantity / u.pix, m_171_l15.wavelength, error_table=error_table)

/home/nabil/.micromamba/envs/solar/lib/python3.13/site-packages/astropy/units/quantity.py:658: RuntimeWarning: invalid value encountered in sqrt
  result = super().__array_ufunc__(function, method, *arrays, **kwargs)

To easily visualize the spatial distribution of these uncertainties, we can create a new sunpy.map.Map object from these errors.

[38]:

m_171_errors = sunpy.map.Map(errors_171.value, m_171_l15.meta)

[39]:

m_171_errors.peek(norm=ImageNormalize(vmax=50))

We can also use the include_chianti flag to understand how the presence of atomic data errors from CHIANTI affect the uncertainty.

[40]:

errors_171_chianti = estimate_error(
    m_171_l15.quantity / u.pix, m_171_l15.wavelength, include_chianti=True, error_table=error_table
)

/home/nabil/.micromamba/envs/solar/lib/python3.13/site-packages/astropy/units/quantity.py:658: RuntimeWarning: invalid value encountered in sqrt
  result = super().__array_ufunc__(function, method, *arrays, **kwargs)

Similarly, we can look at how the photometric calibration, using the calibration from EVE, affects the uncertainties.

[41]:

errors_171_eve = estimate_error(
    m_171_l15.quantity / u.pix, m_171_l15.wavelength, include_eve=True, error_table=error_table
)

We can then look at the distribution of these uncertainties with these different combinations of keyword arguments to understand how these different options affect the resulting distribution of errors. Note that the inclusion of the atomic data results in the largest errors because there is an additional 25% (for 171 Å) uncertainty assumed on the input intensities.

[42]:

hist_params = {"bins": np.logspace(0, 4, 50), "histtype": "step", "log": True}
plt.hist(errors_171.value.flatten(), **hist_params, label="Nominal")
plt.hist(errors_171_chianti.value.flatten(), **hist_params, label="CHIANTI")
plt.hist(errors_171_eve.value.flatten(), **hist_params, label="Photometric (EVE)")
plt.xlabel("Uncertainty [ct/pix]")
plt.ylabel("Number of Pixels")
plt.xscale("log")
plt.legend(frameon=False)

[42]:

<matplotlib.legend.Legend at 0x70ad8dd0f9d0>

Exposure Time Normalization

For normalizing images by the exposure time:

[43]:

m_171_norm = m_171_l15 / m_171_l15.exposure_time

This option still properly accounts for the change in units but does not reset the exposure time.

[44]:

print(m_171_norm.unit)

DN / s

[45]:

print(m_171_norm.exposure_time)

2.000169 s

Wavelength Response Functions

Lastly, the aiapy.response subpackage includes the Channel object for computing the wavelength response function of each channel as well as providing various pieces of metadata for each channel.

[46]:

c = Channel(m_211.wavelength)

[47]:

print(c.channel)
print(c.telescope_number)

211.0 Angstrom
2

The wavelength response is a combination of the (wavelength-dependent) effective area and the gain of the telescope such that \(R(\lambda,t) = A_{eff}(\lambda)G(\lambda)\). According to Section 2 of Boerner et al. (2012), the effective area is given by,

\[A_{eff}(\lambda) = A_{geo}R_P(\lambda)R_S(\lambda)T_E(\lambda)T_F(\lambda)D(\lambda)Q(\lambda),\]

where \(A_{geo}\) is the geometrical collecting area, \(R_{P,S}\) are the reflectances of the primary and secondary mirrors, respectively, \(T_{E,F}\) are the transmission efficiency of the entrance and focal-plane filters, respectively, \(D\) is the contaminant transmittance of the optics and \(Q\) is the quantum efficiency of the CCD.

We can compute the wavelength response for the 211 Å channel using the following method.

[48]:

correction_table = get_correction_table("JSOC")
r = c.wavelength_response(correction_table=correction_table)

Each of the individual components of the effective area are also available as properties on the Channel object.

[49]:

print(c.geometrical_collecting_area)
print(c.primary_reflectance)
print(c.focal_plane_filter_efficiency)
print(c.contamination)
print(c.quantum_efficiency)
print(c.gain)

83.0 cm2
[1.0735452e-05 1.0784249e-05 1.0833047e-05 ... 2.8031474e-01 2.8032130e-01
 2.8032786e-01]
[5.95474e-01 5.93567e-01 5.91666e-01 ... 3.08021e-05 3.06914e-05
 3.05812e-05]
[0.96578515 0.96544755 0.9651096  ... 0.4387496  0.4387496  0.4387496 ]
[0.8513821  0.8509894  0.8506473  ... 0.09263941 0.09265867 0.09267791]
[7.42477    7.39518913 7.36584303 ... 0.20628946 0.20626652 0.20624361] DN / ph

The .wavelength_response method also includes several keyword arguments for including the effects of degradation and EVE cross-calibration in the computation of the wavelength response function.

[ ]:

r_time = c.wavelength_response(obstime=m_211.date, correction_table=correction_table)
r_time_eve = c.wavelength_response(obstime=m_211.date, include_eve_correction=True, correction_table=correction_table)

[61]:

plt.plot(c.wavelength, r, label="Uncorrected")
plt.plot(c.wavelength, r_time, label="Degradation")
plt.plot(c.wavelength, r_time_eve, label="Degradation + EVE")
plt.xlim([180, 240])
plt.ylim([0, 1.75])
plt.xlabel(r"$\lambda$ [Å]")
plt.ylabel(rf"$R(\lambda)$ [{r.unit.to_string(format='latex_inline')}]")
plt.legend(loc=2, frameon=False)

[61]:

<matplotlib.legend.Legend at 0x70ad6e166c10>

Note that these wavelength responses, when combined with atomic data (e.g. from CHIANTI), can be used to compute the temperature response functions of each AIA channel using Section 2.5 of Boerner et al. (2012). More information about this subpackage can be found in the Computing wavelength response functions example in the aiapy example gallery.

Resources

In addition to this blog post, there are also a number of resources for learning more about aiapy:

aiapy is under active development and, just as with the core sunpy package and all other affilitated packages, contributions from the community are always welcome. To report a bug, request a feature, or simply ask a question, feel free to open an issue on GitLab.

SunPy Awarded NASA Grant

2021-11-03T00:00:00+00:00

As part of the NASA “Open Source Tools, Frameworks, and Libraries” program, the SunPy proposal “Strengthening the Foundations of the SunPy Ecosystem” has been funded for the next three years.

This proposal was submitted in January 2021, and sought funding for both the core sunpy package and wider goals of the project. After early community input was collected, Albert Shih led the proposal itself, with NASA GSFC as the primary institution. The proposed effort focuses on three main areas:

Improving the technical infrastructure.

This focuses on the tools that the wider project needs to scale up as we adopt more coordinated and affiliated packages. To support multiple active projects with a common development workflow, more tooling and infrastructure is needed, both for maintainers of existing packages and developers new to writing affiliated packages. As part of this work templates for setting up new packages will be improved, documentation will be written, and new automation and monitoring bots will be developed to ease the burden on the SunPy maintainers.
Augmenting science-enabling functionality.

This section aims to improve the tools provided by the core package to better support future challenges in solar physics data. The two main components of this are supporting multi-point observations with improvements to the sunpy.coordinates package, and improving the support for very large datasets in sunpy and sponsored packages by leveraging the scaling and parallelism tools available from Dask.
Providing training and outreach

This section will improve example galleries across both sunpy core and affiliated packages, develop training materials for solar physics graduate students, and perform outreach activities with instrument teams and package developers.

What will the money pay for?

The money awarded to this proposal will pay for:

Albert Shih will manage the project and augment parts of sunpy.coordinates.
Stuart Mumford will work on infrastructure for all sponsored and affiliated packages as discussed in sunpy/sunpy-project#2.
Will Barnes will work on the training/outreach and large-dataset-support components of the project.

Finally

This is a major milestone for SunPy - it is the first significant amount of funding dedicated to directly working on the long term sustainability of the project. This funding will help to significantly grow SunPy, both the core package and the ecosystem of associated packages.

October Release Announcements

2021-10-29T00:00:00+00:00

Today we have a quadruple release announcement for you! It’s been a very busy week in SunPy land getting both the core and ndcube releases over the line; I hope you enjoy them!

sunpy 3.1

The next scheduled release of the sunpy core package actually comes a week early according to our release calendar, but we couldn’t resist bringing you this end of month release party.

The 3.1 release contains many new features and improvements, having merged 145 pull requests from 16 people including 8 new contributors.

Some of the highlights from the What’s New in SunPy 3.1? page are:

Increased in-situ data support
A new limb drawing function
A new WISPR map source
Propagating solar-surface coordinates in time
Convenient reprojection of maps
JSOC keyword filtering with Fido
Arithmetic operations with maps

My personal favourites being the new limb drawing function and the reproject_to method on all maps.

ndcube 2.0

Even more excitingly, for me personally, is the long awaited 2.0 release of ndcube.

ndcube is the sunpy sponsored package for working with multi-dimensional data with an associated WCS object. For an introduction to ndcube and motivation for the 2.0 rework see An introduction to ndcube.

We started working on ndcube 2.0 back in 2019 when we had Yash Sharma do the initial work of porting the code base to use the APE 14 APIs as his Google Summer of Code project. Little did any of us think that it would take over two years from that point to hit the final release.

The road to ndcube 2.0 has been long and arduous; all involved have put in a lot of work and what we have made is something that I am personally very proud of. ndcube 2.0 would not have happened without Dan Ryan, who as the co-maintainer for ndcube has spent many hours on video calls with me designing APIs, and many more hours besides working out problems you would only encounter in one in a million WCS objects! I would also be remiss if I didn’t specifically thank the DKIST data center who have, by paying for my time, funded a lot of the development work on ndcube 2.0, even when the scope increased beyond their direct requirements.

I hope that many packages will start using ndcube 2.0 as the base for their code. There are already many who have helped to shape the release such as the DKIST User Tools, specutils and sunraster as well as others I know are in the works.

For all the gory details of the changes in ndcube 2.0 see the changelog.

pfsspy 1.0

The SunPy-affiliated potential field source surface extrapolation package pfsspy maintained by David Stansby has had a 1.0 release, which includes promises of API stability. You can read about all the changes in the changelog.

aiapy 0.6

The SunPy-affiliated aiapy package, which provides tools for working with SDO/AIA data and is maintained by the AIA instrument team, has a new 0.6 release which includes support for sunpy core 3.1. In addition to a number of bug fixes, v0.6 also includes the ability to calculate uncertainties on AIA images via the estimate_error function. A complete list of changes is available in the changelog.

The What, Why, and How of SunPy Affiliated Packages

2020-08-18T00:00:00+00:00

What are SunPy affiliated packages?

SunPy affiliated packages are well-maintained, open-source software packages that build upon or extend the functionality of the SunPy core library. The main idea behind affiliated packages is to provide additional tools and functionality to the solar physics community that is considered outside the scope of the SunPy core library, but builds upon it. Affiliated packages are aimed to integrate well with the SunPy ecosystem and together with the core library offer a diverse software ecosystem for which to perform solar data analysis. Affiliated packages are reviewed and registered with the SunPy project.

There are two types of affiliated packages - sponsored and non-sponsored. A sponsored affiliated package is an affiliated package that is maintained and overseen by the SunPy project, examples of these include sunpy core and ndcube. Non-sponsored affiliated packages on the other hand remain under the control and maintenance of the original developers, but must meet a set of standards. Examples of non-sponsored affiliated packages would be instrument-specific affiliated packages such as AIApy.

In this blog post we want to outline our new streamline process for becoming an affiliated package and we want to encourage package developers, particularly instrument teams interested in developing a Python package, to submit their package for review for SunPy affiliate status!

Why would a package want to become an affiliated package?

SunPy supports the development of affiliated packages through our community development efforts and by providing our package template. The SunPy project will also ensure that affiliated packages are publicized to encourage community development. For example, affiliated packages are listed on the SunPy website, and SunPy adverstises them at conferences and workshops.

Call to instrument/mission teams!

We invite instrument and mission teams that are interested in developing a Python package to become a SunPy affiliated package. Integrating mission/instrument-specific software within the SunPy ecosystem will be of great benefit to the solar community and will allow for a simplified and unified process for users to perform solar data analysis. For example, a package may consist of instrument-specific calibration routines which would be outside the scope of SunPy core package, but could build upon the functionality provided by core such as the data containers and file readers. Developing a mission/instrument specific package as a SunPy affiliated package will allow the code to live and integrate within the SunPy ecosystem.

Furthermore, for mission/instrument teams that would like to begin developing specific Python software, the SunPy project offers support through our package template and community resources for the package to be ‘incubated’ until it is at the level of affiliate status. We would really like to encourage instrument/mission teams to reach out!

Package template

The SunPy project is currently developing a package template that is designed to help developers create new Python packages within the SunPy ecosystem. The template is still in development, and we are working towards creating more documentation to make it accessible for developers to use. The idea is that this template will provide a simplified way to standardized packaging, testing, and documentation using the same framework as the SunPy core package. The SunPy package template will make it easier for package developers to meet the standards required by SunPy to become an affiliated package!

How does a package become an affiliated package?

To become a SunPy affiliated package, a set of criteria need to be met to ensure that affiliated packages provide useful functionality to the community at a standard of quality similar to the core SunPy package. We now have a new review process for becoming an affiliated package, aimed at being open, approachable and streamlined!

The reviews are performed as an open process through GitHub issues on the https://github.com/sunpy/sunpy.org repository. To submit a package for review to become an affiliated package, an issue should be opened on this repository. We now have an issue template for submitting a package for review to make it easier to provide all the required information and to start a dialogue about the process.

Once an issue is open, the Affiliate Package Liaison will check that the package meets the baseline requirements of an affiliated package. These include checking to see that it is compatible with the SunPy Code of Conduct, has an appropriate licence, provides a Python interface, is on PyPI, and is useful to the solar physics community. Following this, a reviewer independent to the package is assigned and a review is undertaken based on the criteria listed below.

The review criteria that are assessed are listed as follows with the associated gradings:

Functionality Does the package provide functionality that is useful and relevant to the solar physics community?
Integration - Does that package integrate well within the SunPy ecosystem? for example does it make use of all appropriate features in the core library, including the applicable data structures and dependencies?
Documentation - Does the package have satisfactory documentation that is up to the standard of the core sunpy library including a user guide and examples?
Testing - Does the package use a testing suite to verify the code functionality within its codebase? Does it provide integration tests?
Duplication - Does the package duplicate functionality within sunpy core or any of the other affiliated packages or other relevant packages?
Community - Is the package openly developed with active input and engagement from users within the community? Does the package have a code of conduct and does this reflect the same values as the SunPy project?
Development Status - Is the package well maintained? Are contribution responded to by developers? Is the package still under development with large API changes?

These criteria are reviewed on a ‘traffic light’ system and ranked ‘green’, ‘orange’, or ‘red’ based on the submitted package. For full details of how each of criteria are ranked please check out the Review Criteria.

For a submitted package to be accepted to affiliate package status, the package must be ‘green’ in Functionality, and one other category. It must also not list any ‘red’ scores.

If the package in its current state does not pass the criteria, it can be listed as Provisional once it does not list ‘red’ in the Functionality, Duplication or Community criteria. The idea is that the package can be listed as Provisional as it is working towards addressing the criteria for which it did not pass the review in.

We would really like to encourage packages interested in becoming an affiliated SunPy package to please submit an issue and open up a review dialogue. We will have an Affiliated Package Liaison that will help you through each step of this process :). Even if you are unsure about whether you want to submit a package, please feel free to open an issue and informally discuss your package.

Reach out!

If you are a developer of a package that you think fits nicely into the SunPy ecosystem and will of benefit to the solar physics community and want to chat to us about it please reach out! This can be of course regardless of how far along the package is - from concept to maturity! Join us our live chat Matrix room or join in on the SunPy weekly community meetings which occur each Wednesday (see our calendar for the time in your timezone) and are hosted on jitsi.

Blog

Artemis II Solar Eclipse

Fitting Coordinate Information

Finding the position of Artemis II

Moon Limb Fitting

Calculating Image Scale

Fitting Roll Angle

Fitting Lens Distortion

Overplotting Coronagraph Images

Anaconda packages are not “free”

Glossary

What is free to use?

What is not free to use?

What is the alternative?

I have Anaconda or Anaconda Navigator already - What should I do?

SunPy Development Team wins a NASA Group Achievement Award!

Process Your Solar Eclipse Photos with SunPy!

Read in Your Eclipse Photo

Find the Moon

Make a SunPy Map

Find a Star

Combine your Image with NASA Data

Conclusion

pyOpenSci and sunpy

The future of time series data in sunpy

User requirements

Existing options for a data container

What do other projects use?

What datasets does sunpy currently support?

Evaluating options

numpy.ndarray

pandas.DataFrame

xarray.DataArray

astropy.timeseries.TimeSeries

NDCube

Initial recommendations

Putting astropy objects in xarray structures

aiapy: A SunPy affiliated package for analyzing data from the Atmospheric Imaging Assembly

Obtaining AIA Data

PSF Deconvolution

Respiking Level 1 Images

Transforming Level 1 Images to Level 1.5

Degradation Correction

Computing Uncertainties

Exposure Time Normalization

Wavelength Response Functions

Resources

SunPy Awarded NASA Grant

What will the money pay for?

Finally

October Release Announcements

sunpy 3.1

ndcube 2.0

pfsspy 1.0

aiapy 0.6

The What, Why, and How of SunPy Affiliated Packages

What are SunPy affiliated packages?

Why would a package want to become an affiliated package?

Call to instrument/mission teams!

Package template

How does a package become an affiliated package?

Reach out!

Make a SunPy `Map`

`numpy.ndarray`

`pandas.DataFrame`

`xarray.DataArray`

`astropy.timeseries.TimeSeries`

`NDCube`

Putting `astropy` objects in `xarray` structures