!yNLbWuxtZEZoUZYwKG:nixos.org

Nix Geospatial Team

37 Members
Nix Geospatial packages maintenance. Team board - https://github.com/orgs/NixOS/projects/47/views/18 Servers

Load older messages


SenderMessageTime
21 Feb 2024
@vengmark2:matrix.orgl0b0
In reply to @imincik:matrix.org
What about opening issue in GDAL ?
* The first thing someone is going to ask is how to reproduce, and I think I'll need some help with that. There are a bajillion options, and I'm not very familiar with cmake, and even less familiar with GDAL. I'd be surprised if they'd be OK with running nix-build to test it, but I could start with that.
23:46:58
@vengmark2:matrix.orgl0b0 Looks like there's already a relevant issue. 23:48:01
@vengmark2:matrix.orgl0b0 * Looks like there's already a relevant issue. I'll try building libtiff with lerc to see if that fixes it… 23:50:47
22 Feb 2024
@vengmark2:matrix.orgl0b0Is anyone familiar enough with libtiff to create some tests? I feel like if LERC support should be added to libtiff then it should be tested in the libtiff package, not in GDAL.00:05:25
@imincik:matrix.orgIvan Mincik (imincik)
In reply to @vengmark2:matrix.org
Looks like there's already a relevant issue. I'll try building libtiff with lerc to see if that fixes it…
Is it working with https://github.com/NixOS/nixpkgs/pull/290556
09:17:53
@imincik:matrix.orgIvan Mincik (imincik)
In reply to @vengmark2:matrix.org
Looks like there's already a relevant issue. I'll try building libtiff with lerc to see if that fixes it…
* Is it working with https://github.com/NixOS/nixpkgs/pull/290556 ?
09:17:57
@vengmark2:matrix.orgl0b0Yep, that's how I verified the libtiff change. There's no libtiff test to verify that LERC support has been enabled yet.09:18:44
@vengmark2:matrix.orgl0b0

You should be able to verify with

git checkout l0b0/libtiff-lerc-support
git cherry-pick l0b0/gdal-test-separation
git cherry-pick l0b0/gdal-lerc-support
nix-build --attr pkgs.gdal.passthru.tests
09:24:05
@vengmark2:matrix.orgl0b0 *

You should be able to verify with

git checkout l0b0/libtiff-lerc-support
git cherry-pick l0b0/gdal-test-separation
git cherry-pick l0b0/gdal-lerc-support
nix-build --attr pkgs.gdal.passthru.tests.compress-lerc
09:24:45
@vengmark2:matrix.orgl0b0 *

You should be able to verify with

git checkout l0b0/libtiff-lerc-support
git cherry-pick l0b0/gdal-test-separation
git cherry-pick l0b0/gdal-lerc-support
nix-build --attr pkgs.gdal.passthru.tests.compress-lerc

(It takes forever to build, though.)

09:25:13
@imincik:matrix.orgIvan Mincik (imincik)

What about trying tiffinfo to check which compression is enabled

./result-bin/bin/tiffinfo ~/Projects/osgeo/qgis/code/tests/testdata/raster_layer.tiffTIFFReadDirectory: Warning, Unknown field with tag 33550 (0x830e) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 33922 (0x8482) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 34735 (0x87af) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 34736 (0x87b0) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 34737 (0x87b1) encountered.
=== TIFF directory 0 ===
TIFF Directory at offset 0x67b86 (424838)
  Image Width: 337 Image Length: 315
  Resolution: 36, 36 pixels/inch
  Bits/Sample: 8
  Compression Scheme: None
  Photometric Interpretation: RGB color
  Extra Samples: 1<unassoc-alpha>
  Orientation: row 0 top, col 0 lhs
  Samples/Pixel: 4
  Rows/Strip: 6
  Planar Configuration: single image plane
  Tag 33550: 0.146439,0.146439,0.000000
  Tag 33922: 0.000000,0.000000,0.000000,-124.091282,56.207996,0.000000
  Tag 34735: 1,1,0,7,1024,0,1,2,1025,0,1,1,2048,0,1,4326,2049,34737,7,0,2054,0,1,9102,2057,34736,1,1,2059,34736,1,0
  Tag 34736: 298.257224,6378137.000000
  Tag 34737: WGS 84|
09:25:42
@imincik:matrix.orgIvan Mincik (imincik) tiffinfo contains Compression Scheme: None field 09:26:26
@vengmark2:matrix.orgl0b0 I did verify using that, but I don't know of any good TIFFs I could use for the test. Ideally I'd use libtiff to create a 1x1 LERC TIFF and then read its info, but libtiff doesn't seem to come with CLI tools to create TIFFs. 09:26:50
@vengmark2:matrix.orgl0b0 * I did verify using that (or one of the other executables included in libtiff; I don't remember), but I don't know of any good TIFFs I could use for the test. Ideally I'd use libtiff to create a 1x1 LERC TIFF and then read its info, but libtiff doesn't seem to come with CLI tools to create TIFFs. 09:27:51
@vengmark2:matrix.orgl0b0Thank you for the help, by the way!09:28:30
@imincik:matrix.orgIvan Mincik (imincik)
In reply to @vengmark2:matrix.org
I did verify using that (or one of the other executables included in libtiff; I don't remember), but I don't know of any good TIFFs I could use for the test. Ideally I'd use libtiff to create a 1x1 LERC TIFF and then read its info, but libtiff doesn't seem to come with CLI tools to create TIFFs.
Then use gdal to create the tiff file.
09:30:20
@vengmark2:matrix.orgl0b0I was going to say that would take 100x the build time, but of course the libtiff tests already include the GDAL test suite. 🤔 OK, I'll give it a shot.09:31:09
@vengmark2:matrix.orgl0b0 * I was going to say that would take 100x the build time, but of course the libtiff tests already include the GDAL test suite. 🤔 OK, I'll give it a shot. Tomorrow. 💤09:31:49
@imincik:matrix.orgIvan Mincik (imincik)
In reply to @imincik:matrix.org
Then use gdal to create the tiff file.

Something like

gdal_create -outsize 10 10 -co COMPRESS=LERC_DEFLATE test.tif
09:34:44
@imincik:matrix.orgIvan Mincik (imincik)

gdalinfo works as well

with gdal: Test LERC support #290292 I am able to do:

nix-build -A gdal

./result/bin/gdal_create -outsize 10 10 -co COMPRESS=LERC_DEFLATE test.tif

./result/bin/gdalinfo test.tif
Driver: GTiff/GeoTIFF
Files: test.tif
Size is 10, 10
Image Structure Metadata:
  COMPRESSION=LERC_DEFLATE
  INTERLEAVE=BAND
  LERC_VERSION=2.4
Corner Coordinates:
Upper Left  (    0.0,    0.0)
Lower Left  (    0.0,   10.0)
Upper Right (   10.0,    0.0)
Lower Right (   10.0,   10.0)
Center      (    5.0,    5.0)
Band 1 Block=10x10 Type=Byte, ColorInterp=Gray
09:52:58
@imincik:matrix.orgIvan Mincik (imincik)I've submitted a Nix talk for FOSS4G Europe conference. Please vote for the talk if you want to help me to be accepted - https://talks.osgeo.org/foss4g-europe-2024/p/voting/signup . Talk title is "Can we use Nix as a default way of distributing geospatial software ?" (second page of the list). Thanks ! 12:34:36
@imincik:matrix.orgIvan Mincik (imincik) New approach of proj-data integration to proj - https://github.com/NixOS/nixpkgs/pull/290643 . I am finally happy with that. 17:29:23
@vengmark2:matrix.orgl0b0
In reply to @imincik:matrix.org
So many issues in those few instructions in elevation README :)
Do you have some specific suggestions? I'd love to improve this process for the team.
20:30:45
@vengmark2:matrix.orgl0b0 Would it be useful to add enable* ? true parameters for each of the optional dependencies, in case someone wants a minimal libtiff? Or is that too much control? 20:40:16
@imincik:matrix.orgIvan Mincik (imincik)
In reply to @vengmark2:matrix.org
Would it be useful to add enable* ? true parameters for each of the optional dependencies, in case someone wants a minimal libtiff? Or is that too much control?
Not for now. Not until somebody needs that.
20:41:55
23 Feb 2024
@timlinux:matrix.orgTim Sutton joined the room.14:34:45
6 Mar 2024
@vengmark2:matrix.orgl0b0 Here's a bit of a conundrum: poetry run pytest -s scripts/stac/imagery/tests/collection_test.py::test_capture_area_added in this branch get different results on NixOS unstable and Ubuntu 22.04. I've narrowed it down to this line. It returns almost identical polygons, with one extra vertex generated on Ubuntu. 22:39:12
@vengmark2:matrix.orgl0b0 * Here's a bit of a conundrum: poetry run pytest -s scripts/stac/imagery/tests/collection_test.py::test_capture_area_added in this branch gets different results on NixOS unstable and Ubuntu 22.04. I've narrowed it down to this line. It returns almost identical polygons, with one extra vertex generated on Ubuntu. 22:39:22
@vengmark2:matrix.orgl0b0 * Here's a bit of a conundrum: poetry run pytest -s scripts/stac/imagery/tests/collection_test.py::test_capture_area_added in this branch gets different results on NixOS unstable and Ubuntu 22.04. I've narrowed it down to this line. It returns almost identical polygons, with one extra vertex generated on Ubuntu. The shapely docs don't mention anything relevant, but I'd like to understand why this is happening to avoid flaky tests. 22:42:31
7 Mar 2024
@vengmark2:matrix.orgl0b0 *

Here's a bit of a conundrum: poetry run pytest -s scripts/stac/imagery/tests/collection_test.py::test_capture_area_added in this branch gets different results on NixOS unstable and Ubuntu 22.04. I've narrowed it down to this line. It returns almost identical polygons, with one extra vertex generated on Ubuntu. The shapely docs don't mention anything relevant, but I'd like to understand why this is happening to avoid flaky tests.

Some more investigation reveals that the additional vertex seems to be the midpoint between two other vertices, and should've been removed by simplify.

00:11:58

Show newer messages


Back to Room ListRoom Version: 10