{ "cells": [ { "cell_type": "markdown", "id": "1c58cff8", "metadata": {}, "source": [ "# Walkthrough\n", "\n", "> *To get it, you first see it, and then let it go*\n", "\n", "In this tutorial πŸ§‘β€πŸ«, we'll step through an Earth Observation πŸ›°οΈ data pipeline\n", "using ``torchdata`` and by the end of this lesson, you should be able to:\n", "- Find Cloud-Optimized GeoTIFFs (COGs) from STAC catalogs πŸ₯ž\n", "- Construct a DataPipe that iteratively reads several COGs in a stream 🌊\n", "- Loop through batches of images in a DataPipe with a DataLoader πŸ‹οΈ\n", "\n", "## πŸŽ‰ **Getting started**\n", "\n", "These are the tools πŸ› οΈ you'll need." ] }, { "cell_type": "code", "execution_count": 1, "id": "237f9ec0", "metadata": {}, "outputs": [], "source": [ "# Geospatial libraries\n", "import pystac\n", "import planetary_computer\n", "import rioxarray\n", "# Deep Learning libraries\n", "import torch\n", "import torchdata\n", "import zen3geo" ] }, { "cell_type": "markdown", "id": "2c24c07b", "metadata": {}, "source": [ "Just to make sure we’re on the same page πŸ“ƒ,\n", "let’s check that we’ve got compatible versions installed." ] }, { "cell_type": "code", "execution_count": 2, "id": "32728b15", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "pystac version: 1.4.0\n", "planetary-computer version: 0.4.6\n", "torch version: 1.11.0+cu102\n", "torchdata version: 0.3.0\n", "zen3geo version: 0.1.0+4d1fa95\n", "rioxarray (0.11.1) deps:\n", " rasterio: 1.2.10\n", " xarray: 2022.3.0\n", " GDAL: 3.3.2\n", " GEOS: None\n", " PROJ: None\n", " PROJ DATA: None\n", " GDAL DATA: None\n", "\n", "Other python deps:\n", " scipy: None\n", " pyproj: 3.3.1\n", "\n", "System:\n", " python: 3.9.13 (main, Jun 1 2022, 20:52:48) [GCC 11.2.0]\n", "executable: /home/docs/checkouts/readthedocs.org/user_builds/zen3geo/envs/v0.1.0/bin/python\n", " machine: Linux-5.15.0-1004-aws-x86_64-with-glibc2.35\n" ] } ], "source": [ "print(f\"pystac version: {pystac.__version__}\")\n", "print(f\"planetary-computer version: {planetary_computer.__version__}\")\n", "print(f\"torch version: {torch.__version__}\")\n", "\n", "print(f\"torchdata version: {torchdata.__version__}\")\n", "print(f\"zen3geo version: {zen3geo.__version__}\")\n", "rioxarray.show_versions()" ] }, { "cell_type": "markdown", "id": "af2436aa", "metadata": {}, "source": [ "## 0️⃣ Find [Cloud-Optimized GeoTIFFs](https://www.cogeo.org) πŸ—ΊοΈ\n", "\n", "Let's get some optical satellite data using [STAC](https://stacspec.org)!\n", "How about Sentinel-2 L2A data over Singapore πŸ‡ΈπŸ‡¬?\n", "\n", "πŸ”— Links:\n", "- [Official Sentinel-2 description page at ESA](https://sentinel.esa.int/web/sentinel/missions/sentinel-2)\n", "- [Microsoft Planetary Computer STAC Explorer](https://planetarycomputer.microsoft.com/explore?c=103.8152%2C1.3338&z=10.08&v=2&d=sentinel-2-l2a&s=false%3A%3A100%3A%3Atrue&ae=0&m=cql%3A2ff1401acb50731fa0a6d1e2a46f3064&r=Natural+color)\n", "- [AWS Sentinel-2 Cloud-Optimized GeoTIFFs](https://registry.opendata.aws/sentinel-2-l2a-cogs)" ] }, { "cell_type": "code", "execution_count": 3, "id": "f794cc0d", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "item_url = \"https://planetarycomputer.microsoft.com/api/stac/v1/collections/sentinel-2-l2a/items/S2A_MSIL2A_20220115T032101_R118_T48NUG_20220115T170435\"\n", "\n", "# Load the individual item metadata and sign the assets\n", "item = pystac.Item.from_file(item_url)\n", "signed_item = planetary_computer.sign(item)\n", "signed_item" ] }, { "cell_type": "markdown", "id": "4dd43b04", "metadata": {}, "source": [ "### Inspect one of the data assets 🍱\n", "\n", "The Sentinel-2 STAC item contains several assets.\n", "These include different 🌈 bands (e.g. 'B02', 'B03', 'B04').\n", "Let's just use the 'visual' product for now which includes the RGB bands." ] }, { "cell_type": "code", "execution_count": 4, "id": "9e5906ee", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
<xarray.DataArray (band: 3, y: 10980, x: 10980)>\n",
       "[361681200 values with dtype=uint8]\n",
       "Coordinates:\n",
       "  * band         (band) int64 1 2 3\n",
       "  * x            (x) float64 3e+05 3e+05 3e+05 ... 4.098e+05 4.098e+05 4.098e+05\n",
       "  * y            (y) float64 2e+05 2e+05 2e+05 ... 9.026e+04 9.026e+04 9.024e+04\n",
       "    spatial_ref  int64 0\n",
       "Attributes:\n",
       "    _FillValue:    0.0\n",
       "    scale_factor:  1.0\n",
       "    add_offset:    0.0
" ], "text/plain": [ "\n", "[361681200 values with dtype=uint8]\n", "Coordinates:\n", " * band (band) int64 1 2 3\n", " * x (x) float64 3e+05 3e+05 3e+05 ... 4.098e+05 4.098e+05 4.098e+05\n", " * y (y) float64 2e+05 2e+05 2e+05 ... 9.026e+04 9.026e+04 9.024e+04\n", " spatial_ref int64 0\n", "Attributes:\n", " _FillValue: 0.0\n", " scale_factor: 1.0\n", " add_offset: 0.0" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "url: str = signed_item.assets[\"visual\"].href\n", "da = rioxarray.open_rasterio(filename=url)\n", "da" ] }, { "cell_type": "markdown", "id": "3a91cef0", "metadata": {}, "source": [ "This is how the Sentinel-2 image looks like over Singapore on 15 Jan 2022.\n", "\n", "![Sentinel-2 image over Singapore on 20220115](https://planetarycomputer.microsoft.com/api/data/v1/item/preview.png?collection=sentinel-2-l2a&item=S2A_MSIL2A_20220115T032101_R118_T48NUG_20220115T170435&assets=visual&asset_bidx=visual%7C1%2C2%2C3&nodata=0)\n", "\n", "## 1️⃣ Construct [DataPipe](https://github.com/pytorch/data/tree/v0.3.0#what-are-datapipes) πŸ“‘\n", "\n", "A torch `DataPipe` is a way of composing data (rather than inheriting data).\n", "Yes, I don't know what it really means either, so here's some extra reading.\n", "\n", "πŸ”– References:\n", "- https://pytorch.org/blog/pytorch-1.11-released/#introducing-torchdata\n", "- https://github.com/pytorch/data/tree/v0.3.0#what-are-datapipes\n", "- https://realpython.com/inheritance-composition-python\n", "\n", "### Create an Iterable πŸ“\n", "\n", "Start by wrapping a list of URLs to the Cloud-Optimized GeoTIFF files.\n", "We only have 1 item so we'll use ``[url]``, but if you have more, you can do\n", "``[url1, url2, url3]``, etc. Pass this iterable list into\n", "{py:class}`torchdata.datapipes.iter.IterableWrapper`:" ] }, { "cell_type": "code", "execution_count": 5, "id": "3b9f42e2", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dp = torchdata.datapipes.iter.IterableWrapper(iterable=[url])\n", "dp" ] }, { "cell_type": "markdown", "id": "253daf47", "metadata": {}, "source": [ "The ``dp`` variable is the DataPipe!\n", "Now to apply some more transformations/functions on it.\n", "\n", "### Read using RioXarrayReader 🌐\n", "\n", "This is where ☯ ``zen3geo`` comes in. We'll be using the\n", "{py:class}`zen3geo.datapipes.rioxarray.RioXarrayReaderIterDataPipe` class, or\n", "rather, the short alias {py:class}`zen3geo.datapipes.RioXarrayReader`.\n", "\n", "Confusingly, there are two ways or forms of applying ``RioXarrayReader``,\n", "a class-based method and a functional method." ] }, { "cell_type": "code", "execution_count": 6, "id": "14d58eff", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Using class constructors\n", "dp_rioxarray = zen3geo.datapipes.RioXarrayReader(source_datapipe=dp)\n", "dp_rioxarray" ] }, { "cell_type": "code", "execution_count": 7, "id": "f0943f6c", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Using functional form (recommended)\n", "dp_rioxarray = dp.read_from_rioxarray()\n", "dp_rioxarray" ] }, { "cell_type": "markdown", "id": "e985fd1c", "metadata": {}, "source": [ "Note that both ways are equivalent (they produce the same IterDataPipe output),\n", "but the latter (functional) form is preferred, see also\n", "https://pytorch.org/data/0.4.0/tutorial.html#registering-datapipes-with-the-functional-api\n", "\n", "What if you don't want the whole Sentinel-2 scene at the full 10m resolution?\n", "Since we're using Cloud-Optimized GeoTIFFs, you could set an ``overview_level``\n", "(following https://corteva.github.io/rioxarray/stable/examples/COG.html)." ] }, { "cell_type": "code", "execution_count": 8, "id": "03a12b23", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dp_rioxarray_zoom3 = dp.read_from_rioxarray(overview_level=3)\n", "dp_rioxarray_zoom3" ] }, { "cell_type": "markdown", "id": "eeaa0c8c", "metadata": {}, "source": [ "Extra keyword arguments will be handled by {py:func}`rioxarray.open_rasterio`\n", "or {py:func}`rasterio.open`.\n", "\n", "```{note}\n", "Other DataPipe classes/functions can be stacked or joined to this basic GeoTIFF\n", "reader. For example, clipping by bounding box or reprojecting to a certain\n", "Coordinate Reference System. If you would like to implement this, check out the\n", "[Contributing Guidelines](./CONTRIBUTING) to get started!\n", "```\n", "\n", "## 2️⃣ Loop through DataPipe βš™οΈ\n", "\n", "A DataPipe describes a flow of information.\n", "Through a series of steps it goes,\n", "as one piece comes in, another might follow.\n", "\n", "### Basic iteration ♻️\n", "\n", "At the most basic level, you could iterate through the DataPipe like so:" ] }, { "cell_type": "code", "execution_count": 9, "id": "6292c4cc", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
<xarray.DataArray (band: 3, y: 687, x: 687)>\n",
       "[1415907 values with dtype=uint8]\n",
       "Coordinates:\n",
       "  * band         (band) int64 1 2 3\n",
       "  * x            (x) float64 3.001e+05 3.002e+05 ... 4.096e+05 4.097e+05\n",
       "  * y            (y) float64 2e+05 1.998e+05 1.996e+05 ... 9.048e+04 9.032e+04\n",
       "    spatial_ref  int64 0\n",
       "Attributes:\n",
       "    _FillValue:    0.0\n",
       "    scale_factor:  1.0\n",
       "    add_offset:    0.0
" ], "text/plain": [ "StreamWrapper<\n", "[1415907 values with dtype=uint8]\n", "Coordinates:\n", " * band (band) int64 1 2 3\n", " * x (x) float64 3.001e+05 3.002e+05 ... 4.096e+05 4.097e+05\n", " * y (y) float64 2e+05 1.998e+05 1.996e+05 ... 9.048e+04 9.032e+04\n", " spatial_ref int64 0\n", "Attributes:\n", " _FillValue: 0.0\n", " scale_factor: 1.0\n", " add_offset: 0.0>" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "it = iter(dp_rioxarray_zoom3)\n", "filename, dataarray = next(it)\n", "dataarray" ] }, { "cell_type": "markdown", "id": "055ea47f", "metadata": {}, "source": [ "Or if you're more familiar with a for-loop, here it is:" ] }, { "cell_type": "code", "execution_count": 10, "id": "d3ffb02c", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "StreamWrapper<\n", "[1415907 values with dtype=uint8]\n", "Coordinates:\n", " * band (band) int64 1 2 3\n", " * x (x) float64 3.001e+05 3.002e+05 ... 4.096e+05 4.097e+05\n", " * y (y) float64 2e+05 1.998e+05 1.996e+05 ... 9.048e+04 9.032e+04\n", " spatial_ref int64 0\n", "Attributes:\n", " _FillValue: 0.0\n", " scale_factor: 1.0\n", " add_offset: 0.0>\n" ] } ], "source": [ "for filename, dataarray in dp_rioxarray_zoom3:\n", " print(dataarray)\n", " # Run model on this data batch" ] }, { "cell_type": "markdown", "id": "c92aff91", "metadata": {}, "source": [ "### Into a DataLoader πŸ‹οΈ\n", "\n", "For the deep learning folks, you might need one extra step.\n", "The {py:class}``xarray.DataArray`` needs to be converted to a tensor.\n", "In the Pytorch world, that can happen via {py:func}``torch.as_tensor``." ] }, { "cell_type": "code", "execution_count": 11, "id": "9319e45d", "metadata": {}, "outputs": [], "source": [ "def fn(da):\n", " return torch.as_tensor(da.data)" ] }, { "cell_type": "markdown", "id": "9e3e5254", "metadata": {}, "source": [ "Using {py:class}`torchdata.datapipes.iter.Mapper`,\n", "we'll apply the tensor conversion function to index 1 of the\n", "``(filename, dataarray)`` tuple." ] }, { "cell_type": "code", "execution_count": 12, "id": "155c7cb3", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dp_tensor = dp_rioxarray_zoom3.map(fn=fn, input_col=1)\n", "dp_tensor" ] }, { "cell_type": "markdown", "id": "35911f28", "metadata": {}, "source": [ "Finally, let's put our DataPipe into a {py:class}`torch.utils.data.DataLoader`!" ] }, { "cell_type": "code", "execution_count": 13, "id": "187d85b3", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "tensor([[[[ 46, 29, 34, ..., 241, 246, 255],\n", " [ 83, 73, 66, ..., 248, 255, 251],\n", " [ 53, 43, 55, ..., 246, 247, 243],\n", " ...,\n", " [101, 101, 104, ..., 78, 179, 83],\n", " [ 99, 103, 105, ..., 68, 60, 45],\n", " [ 95, 103, 102, ..., 50, 34, 42]],\n", "\n", " [[ 58, 24, 44, ..., 255, 255, 255],\n", " [ 60, 51, 57, ..., 255, 255, 254],\n", " [ 47, 22, 47, ..., 255, 255, 255],\n", " ...,\n", " [110, 111, 114, ..., 95, 189, 87],\n", " [110, 113, 113, ..., 85, 62, 48],\n", " [108, 112, 112, ..., 62, 60, 62]],\n", "\n", " [[ 42, 22, 29, ..., 255, 255, 255],\n", " [ 43, 41, 39, ..., 255, 255, 254],\n", " [ 35, 30, 37, ..., 255, 255, 255],\n", " ...,\n", " [ 82, 82, 83, ..., 74, 174, 57],\n", " [ 82, 84, 84, ..., 57, 46, 28],\n", " [ 80, 83, 82, ..., 37, 31, 31]]]], dtype=torch.uint8)\n" ] } ], "source": [ "dataloader = torch.utils.data.DataLoader(dataset=dp_tensor)\n", "for batch in dataloader:\n", " filename, tensor = batch\n", " print(tensor)" ] }, { "cell_type": "markdown", "id": "7d2565da", "metadata": {}, "source": [ "And so it begins πŸŒ„\n", "\n", "---\n", "\n", "That’s all πŸŽ‰! For more information on how to use DataPipes, check out:\n", "\n", "- Tutorial at https://pytorch.org/data/0.4.0/tutorial.html\n", "- Usage examples at https://pytorch.org/data/0.4.0/examples.html\n", "\n", "If you have any questions πŸ™‹, feel free to ask us anything at\n", "https://github.com/weiji14/zen3geo/discussions or visit the Pytorch forums at\n", "https://discuss.pytorch.org/c/data/37.\n", "\n", "Cheers!" ] } ], "metadata": { "jupytext": { "formats": "md:myst", "text_representation": { "extension": ".md", "format_name": "myst" } }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.13" }, "source_map": [ 11, 27, 36, 41, 49, 62, 69, 77, 81, 104, 107, 121, 127, 131, 141, 144, 166, 170, 174, 178, 186, 189, 195, 198, 202, 207 ] }, "nbformat": 4, "nbformat_minor": 5 }