Eye2sky network

Eye2sky network#

This page presents an overview of the Eye2Sky cloud camera network in north-west Germany. Part of the dataset has been made public on Zenodo, and an example of how to retrieve data using Python is provided. See Schmidt et al. (2022) for more information.

“In north-west Germany between Oldenburg, the North Sea coast and the Dutch border, the Institute of Networked Energy Systems operates the unique Eye2Sky cloud camera network as a research infrastructure. The network is growing dynamically and currently consists of around 30 stations. The heart of each station is a cloud camera, also known as an all-sky imager (ASI). This is a commercially available webcam with a fisheye lens supplemented by a ventilation and heating system that ensures optimum image quality even in bad weather. A third of the stations are equipped with a rotating shadowband irradiometer (RSI), other radiation sensors and meteorological sensors for temperature and humidity, for example, to validate and calibrate the algorithms.”

List and map of stations#

The Eye2sky comprises of around 30 stations with different characteristics, which are provided in the table below. Common for all stations is that they are located in north-western Germany as shown on the map below.

init_notebook_modetrusted

Station ID	Place	Domain	Type	Latitude	Longitude	Elevation	Height above ground	Altitude (above sea level)	Installation date
Loading ITables v2.2.2 from the `init_notebook_mode` cell... (need help?)

Make this Notebook Trusted to load map: File -> Trust Notebook

Visualize data#

To visualize the data, the below code cells demonstrate how to plot a single day of GHI data.

_images/cec5d2b1596af311f47f4a2dc0962c13b8335c18b964e3c55c22dc9bc09b36dc.png

Multiple irradiance components#

Some station measure all three standard irradiance components: GHI, DHI, and DNI.

Let’s take a look at measurements from the OLWIN station.

_images/7ac60575866f0cfa268580a707e6f5fde2eb34a997729d92788156a360070814.png

A good way to quality control multi-component irradiance datasets is to compare the measure GHI measurements with the sum of diffuse and direct measurements.

_images/b0baf59f68e7df0279f7b91bf1def944b7741ac15bbbdc148263495a9a5a86be.png

Raw data and QC flags#

In this section we will dive into the quality control flags which are provided with the dataset.

As a first step, we retrieve Eye2sky data with flags:

data = get_eye2sky(
    station='WITTM',
    start='2022-07-21',
    end='2022-07-25',
    file_type='flagged',
)

DNI flags#

par = 'DNI'

# Test Names
qc_name = data[f'{par}.flag'].attrs['name'].split(',')
# Test Codes 
qc_code = data[f'{par}.flag'].attrs['code'].split(',')
# Test actions (how is data that failed tests treated in the cleaned data file)
qc_action = data[f'{par}.flag'].attrs['l1_action'].split(',')
# A description / long name of each test
qc_desc = data[f'{par}.flag'].attrs['description'].split(',')

tests = pd.DataFrame([qc_name, qc_code, qc_action], index=['Name', 'Code', 'Action'])
show(tests)

	0	1	2	3	4	5	6	7	8	9	10	11	12	13
Loading ITables v2.2.2 from the `init_notebook_mode` cell... (need help?)

Timeseries of flag values#

data[f'{par}.flag'].plot()

[<matplotlib.lines.Line2D at 0x7fbc78e79100>]

_images/4374153fd676c8b89f8630f1adeb37efaadb8df30d4426a1120d1999b19cf608.png

Decode the Flags and make them human-readable#

# This function decodes flag decimal value and returns a dictionary of boolean True/False for each single test
def decoding_flgs(dset, test_names):
    # Number of tests (code 0 == valid is not counting)
    tests = test_names[1:]
    ntests = len(tests)
    dec = np.copy(dset)
    flgs = (np.repeat(np.zeros(ntests), len(dec))).reshape(len(dec), ntests)
    for i in range(0, len(dec)):
        for j in range(0, len(flgs[i])):
            flgs[i][j] = dec[i] % 2
            dec[i] = dec[i] // 2
    df = {}
    for i, col in enumerate(tests):
        df[col] = flgs[:,i].astype(bool)
    return df

# Decode also the .test variable. It contains information if the test has been applied to the parameter or not
flags = data[f'{par}.flag']
tests = data[f'{par}.test']
tests = decoding_flgs(tests, qc_desc)
flags = decoding_flgs(flags, qc_desc)

Print some statistics#

# Print statistics
for key, value in flags.items():
    # only if test has been applied
    if all(tests[key]):
        if any(value):
            # Check for flags
            ind = value == True
            # Check for consecutive flags (-> periods)
            ind = np.diff(ind,prepend=False)
            periods = data['time'][ind].values.reshape(int(np.sum(ind)/2),2)
            print(f'QC failed for test "{key}"')
            print(f'Number of data points {len(periods)} ({100*len(periods)/len(data['time']):.2f} %)')
            #for i in range(periods.shape[0]):

QC failed for test "tracking error"
Number of data points 1 (0.01 %)
QC failed for test "two redundant measurements deviate from each other"
Number of data points 5 (0.07 %)
QC failed for test "shading based on horizon file"
Number of data points 10 (0.14 %)

Make a plot for each single test#

_images/5841de77b70cec845752e5437f2590c0979dd6fcc29d459d230ad8e79b3bc538.png

_images/37277c9a779507d7f9f79db7e474bde473563111e8e41b881474b05bd271573d.png

_images/b623fc4d1868e0e04efb647aa02fd00b5a244fdae17144aa556da9613110df6c.png

References#

T. Schmidt, J. Stührenberg, N. Blum, J. Lezaca, A. Hammer and T. Vogt, “A network of all sky imagers (ASI) enabling accurate and high-resolution very short-term forecasts of solar irradiance,” 21st Wind & Solar Integration Workshop (WIW 2022), Hybrid Conference, The Hague, Netherlands, 2022, pp. 372-378, doi: 10.1049/icp.2022.2778.