Dataset Module

This module is used to download anf manipulate the data downloaded from an FTP server

It prepares the data to be used by the other modules

The data is downloaded from the FTP server and stored in a local directory

To import the module

import terravide.src.dataset as dataset

Module Dependencies

The terrvide.src.dataset module has the following dependencies:

multiprocessing.Pool
ftplib.FTP
os
os.path

These modules must be installed before using the functions in this module.

API

FTP_download_lasfile

FTP_download_lasfile(filename, datayear=2017, folderpath='FTP_files/') → None

Downloads a las file from ftp.gis.ny.gov.

Parameters:

filename (str) – The name of the las file to download from the FTP server.
datayear (int) – The year of the data (2017 or 2021). Defaults to 2017.
folderpath (str) – The path to the folder where the file should be downloaded. Defaults to “FTP_files/”.

Returns:

None.

Here is an example usage of the FTP_download_lasfile function:

import terravide.src.dataset as dataset

filename = "myfile.las"
datayear = 2017
folderpath = "/path/to/my/folder/"
dataset.FTP_download_lasfile(filename, datayear, folderpath)

This will download the specified las file from the FTP server and save it to the folder located at folderpath/NYC_2017/, or folderpath/NYC_2021/ if datayear is set to 2021.

FTP_GetFileList

FTP_GetFileList(datayear=2017) → list

Gets a list of all files in the FTP directory of NYC scans for the specified year.

Parameters:: datayear (int) – The year of the data (2017 or 2021). Defaults to 2017.
Returns:: A list of filenames in the FTP server.

Here is an example usage of the FTP_GetFileList function:

import terravide.src.dataset as dataset

datayear = 2017
filenames = dataset.FTP_GetFileList(datayear)
print(filenames)

This will print a list of all files in the FTP directory of NYC scans for the year 2017.

FTP_list_files

FTP_list_files(datayear=2021) → None

Lists all files in the lidar directory of NYC scans for the specified year.

Parameters:: datayear (int) – The year of the data (2017 or 2021). Defaults to 2021.
Returns:: Prints to console.

Here is an example usage of the FTP_list_files function:

import terravide.src.dataset as dataset

datayear = 2021
dataset.FTP_list_files(datayear)

This will list all files in the lidar directory of NYC scans for the year 2021.

Get_filenames

Get_filenames(folder_path: str, year: int) → list

Gets a list of filenames in a folder generated by the FTP_download_lasfile function for the specified year.

Parameters:

folder_path (str) – The path to the folder containing the files.
year (int) – The subfolder name designated by the year of the data (e.g. 2017, 2021).

Returns:

A list of filenames in the specified folder.

Here is an example usage of the Get_filenames function:

import terravide.src.dataset as dataset

folder_path = "/path/to/my/folder"
year = 2021
filenames = dataset.Get_filenames(folder_path, year)
print(filenames)

This will print a list of filenames in the folder located at folder_path/NYC_2021/, which were generated by the FTP_download_lasfile function.

Delete_File

Delete_File(file_path: str) → None

Deletes the file at the specified file path. If the file does not exist, a message is printed indicating that the file was not found.

Parameters:: file_path (str) – The path to the file that should be deleted.
Returns:: None

Here is an example usage of the Delete_File function:

import terravide.src.dataset as dataset

file_path = "/path/to/myfile.txt"
dataset.Delete_File(file_path)

This will delete the file located at file_path, or print a message indicating that the file was not found if it does not exist.