This is the ONS Postcode Directory (ONSPD) for the United Kingdom as at February 2023 in Comma Separated Variable (CSV) and ASCII text (TXT) formats. This file contains the multi CSVs so that postcode areas can be opened in MS Excel. To download the zip file click the Download button. The ONSPD relates both current and terminated postcodes in the United Kingdom to a range of current statutory administrative, electoral, health and other area geographies. It also links postcodes to pre-2002 health areas, 1991 Census enumeration districts for England and Wales, 2001 Census Output Areas (OA) and Super Output Areas (SOA) for England and Wales, 2001 Census OAs and SOAs for Northern Ireland and 2001 Census OAs and Data Zones (DZ) for Scotland. It now contains 2021 Census OAs and SOAs for England and Wales. It helps support the production of area based statistics from postcoded data. The ONSPD is produced by ONS Geography, who provide geographic support to the Office for National Statistics (ONS) and geographic services used by other organisations. The ONSPD is issued quarterly. (File size - 234 MB)NOTE: The 2022 ONSPDs included an incorrect update of the ITL field with two LA changes in Northamptonshire. This error has been corrected from the February 2023 ONSPD.NOTE: There was an issue with the originally published file where some change orders yet to be included in OS Boundary-LineÔ (including The Cumbria (Structural Changes) Order 2022, The North Yorkshire (Structural Changes) Order 2022 and The Somerset (Structural Changes) Order 2022) were mistakenly implemented for terminated postcodes. Version 2 corrects this, so that ward codes E05014171–E05014393 are not yet included. Please note that this product contains Royal Mail, Gridlink, LPS (Northern Ireland), Ordnance Survey and ONS Intellectual Property Rights.

Our UK Postcode Database offers comprehensive postal code data for spatial analysis, including postal and administrative areas. This dataset contains accurate and up-to-date information on all administrative divisions, cities, and zip codes, making it an invaluable resource for various applications such as address capture and validation, map and visualization, reporting and business intelligence (BI), master data management, logistics and supply chain management, and sales and marketing. Our location data packages are available in various formats, including CSV, optimized for seamless integration with popular systems like Esri ArcGIS, Snowflake, QGIS, and more. Product features include fully and accurately geocoded data, multi-language support with address names in local and foreign languages, comprehensive city definitions, and the option to combine map data with UNLOCODE and IATA codes, time zones, and daylight saving times. Companies choose our location databases for their enterprise-grade service, reduction in integration time and cost by 30%, and weekly updates to ensure the highest quality.

This data-set contains all data resources, either directly downloadable via this platform or as links to external databases, to execute the generic modeling tool as described in D5.4

• 0 generic model
  file: d5.4-generic-model_february-2020.pdf
  Report with methodology and source code for the generic model
• 1.1 COPERNICUS
  https://land.copernicus.eu/local/urban-atlas/urban-atlas-2012?tab=download.%20
  The European Urban Atlas provides reliable, inter-comparable, high-resolution land use and land cover data for 800 Functional Urban Area (FUA) for the 2012 reference year in EEA39 countries.
• 1.2a 1km grid shapefiles for NOx emissions in EU
  The shapefiles are: nox.dbf nox.prj nox.shp nox.shx
• 1.2b 1km grid shapefiles for PM10 emissions in EU
  The shapefiles are: pm10.dbf pm10.prj pm10.shp pm10.shx
• 2.1 Eurostat energy consumption data
  https://ec.europa.eu/eurostat/web/energy/data/database?p_p_id=NavTreeportletprod_WAR_NavTreeportletprod_INSTANCE_QAMy7Pe6HwI1&p_p_lifecycle=0&p_p_state=normal&p_p_mode=view&p_p_col_id=column-2&p_p_col_count=1
• 2.2 JODI gas consumption data
  https://www.jodidata.org/gas/database/data-downloads.aspx
  Complete data series for all products, flows and countries, from January 2009 to one month-old can be downloaded, for free, in a beta version of .csv format.
• 2.3 Meteo data UK
  https://www.metoffice.gov.uk/services/data/datapoint/datapoint-products
• 2.4 Meteo data NL
  https://www.knmi.nl/nederland-nu/klimatologie/daggegevens
• 2.5 hourly electricity load profiles
  https://data.open-power-system-data.org/time_series/
• 3.1 fleet data
  file: 200210_summary_fleet_2016.csv
  Summary of historical fleet for EU28+NO by age and vehicle type: Diesel, CNG, Petrol, Electric, PHEV, LPG
• 3.2 modal shares
  file: 200210_summary_shares.csv
  This data set holds the modal share of different modes, for EU cities at level of vehicle kilometers. Modes considered: bus car heavy duty truck >32t heavy duty truck 16-32t heavy duty truck 3_5-7_5t heavy duty truck 7_5-16t light duty truck metro/tram moped motorcycle passenger train slow mode van
• 3.3 emission factors
  file: 200212_summary_copert_ef.xlsx
  Emission factor for passenger cars (in g/km )from COPERT V, urban conditions. Data dimensions: 1. type: diesel, petrol, CNG, LPG, PHEV (in line with fleet data) 2. pollutant: NOx or PM 3. Peak/off-peak: (P/OP) 4. year of built: YOB
• 5.1 Crude death rates
  http://apps.who.int/healthinfo/statistics/mortality/causeofdeath_query/start.php
  the total number of deaths in a country per year per 1,000 people broken down by age cohort and sex
• 5.2 UN population data
  https://population.un.org/wpp/Download/Standard/Population/
  the number of inhabitants at country level broken down by age cohort and sex the time a person has lived broken down by age cohort and sex, and country

This file contains the National Statistics UPRN Lookup (NSUL) for Great Britain as at February 2023. The NSUL relates the Unique Property Reference Number (UPRN) for each GB address from AddressBase® Epoch 99 to a range of current statutory administrative, electoral, health and other statistical geographies via 'best-fit' allocation from 2021 Census output areas (National Parks and Workplace Zones are exempt from 'best-fit' and use 'exact-fit' allocations). The NSUL is produced by ONS Geography, who provide geographic support to the Office for National Statistics (ONS) and geographic services used by other organisations. The NSUL is issued every 6 weeks and is designed to complement the Ordnance Survey AddressBase® product. For further technical information about this file, please refer to the User Guide document contained within the downloadable zip file. Please note that this product contains Royal Mail, Gridlink, Ordnance Survey and ONS Intellectual Property Rights. (File Size – 463 MB)

THIS RESOURCE IS NO LONGER IN SERVICE. Documented on February 28,2023. Database of secondary structure assignments (and much more) for all protein entries in the Protein Data Bank (PDB) and the program that calculates DSSP entries from PDB entries. DSSP is distributed on a basis of trust and instructions are available on the site. * Precompiled executables are also available for Linux and Windows. (The Windows .exe file was compiled under Linux using Mingw32, has never seen a Windows environment and should thus be virus-free. Download the source if you want to be 100% sure.) Under Windows the DSSP output does not make it to the console, so redirect it to a file instead: dsspcmbi source.pdb destination.dssp > messages.txt * Several changes have been made to the DSSP program to solve problems with recent PDB files. These are documented in the source code. * FTP access to the DSSP files resides at the CMBI: ftp.cmbi.kun.nl/pub/molbio/data/dssp or ftp://ftp.ebi.ac.uk/pub/databases/dssp/. If you have problems downloading the DSSP files, it is likely that your FTP program is not able to handle tens of thousands of files in one directory. In this case, install a proper FTP program, for example NCFTP. However, it is recommended that you download DSSP files with the rsync command.

Part of the dissertation Pitch of Voiced Speech in the Short-Time Fourier Transform: Algorithms, Ground Truths, and Evaluation Methods.
© 2020, Bastian Bechtold. All rights reserved.

This dataset contains common speech and noise corpora for evaluating fundamental frequency estimation algorithms as convenient JBOF dataframes. Each corpus is available freely on its own, and allows redistribution:

• CMU-ARCTIC (BSD license) [1]
• FDA (free to download) [2]
• KEELE (free for noncommercial use) [3]
• MOCHA-TIMIT (free for noncommercial use) [4]
• PTDB-TUG (ODBL license) [5]
• NOISEX (free to download) [7]
• QUT-NOISE (CC-BY-SA license) [8]

Additionally, this dataset contains PDAs-0.0.1-py3-none-any.whl, a Python ≥ 3.6 module for Linux, containing several well-known fundamental frequency estimation algorithms:

• AUTOC [9]
• AMDF [10]
• BANA [11]
• CEP [12]
• CREPE [13]
• DIO [14]
• DNN [15]
• KALDI [16]
• MAPS
• MBSC [17]
• NLS [18]
• PEFAC [19]
• PRAAT [20]
• RAPT [21]
• SACC [22]
• SAFE [23]
• SHR [24]
• SIFT [25]
• SRH [26]
• STRAIGHT [27]
• SWIPE [28]
• YAAPT [29]
• YIN [30]

The algorithms are included in their native programming language (Matlab for BANA, DNN, MBSC, NLS, NLS2, PEFAC, RAPT, RNN, SACC, SHR, SRH, STRAIGHT, SWIPE, YAAPT, and YIN; C for KALDI, PRAAT, and SAFE; Python for AMDF, AUTOC, CEP, CREPE, MAPS, and SIFT), and adapted to a common Python interface. AMDF, AUTOC, CEP, and SIFT are our partial re-implementations as no original source code could be found.

All algorithms have been released as open source software, and are covered by their respective licenses.

All of these files are published as part of my dissertation, "Pitch of Voiced Speech in the Short-Time Fourier Transform: Algorithms, Ground Truths, and Evaluation Methods", and in support of the Replication Dataset for Fundamental Frequency Estimation.

References:

1. John Kominek and Alan W Black. CMU ARCTIC database for speech synthesis, 2003.
2. Paul C Bagshaw, Steven Hiller, and Mervyn A Jack. Enhanced Pitch Tracking and the Processing of F0 Contours for Computer Aided Intonation Teaching. In EUROSPEECH, 1993.
3. F Plante, Georg F Meyer, and William A Ainsworth. A Pitch Extraction Reference Database. In Fourth European Conference on Speech Communication and Technology, pages 837–840, Madrid, Spain, 1995.
4. Alan Wrench. MOCHA MultiCHannel Articulatory database: English, November 1999.
5. Gregor Pirker, Michael Wohlmayr, Stefan Petrik, and Franz Pernkopf. A Pitch Tracking Corpus with Evaluation on Multipitch Tracking Scenario. page 4, 2011.
6. John S. Garofolo, Lori F. Lamel, William M. Fisher, Jonathan G. Fiscus, David S. Pallett, Nancy L. Dahlgren, and Victor Zue. TIMIT Acoustic-Phonetic Continuous Speech Corpus, 1993.
7. Andrew Varga and Herman J.M. Steeneken. Assessment for automatic speech recognition: II. NOISEX-92: A database and an experiment to study the effect of additive noise on speech recog- nition systems. Speech Communication, 12(3):247–251, July 1993.
8. David B. Dean, Sridha Sridharan, Robert J. Vogt, and Michael W. Mason. The QUT-NOISE-TIMIT corpus for the evaluation of voice activity detection algorithms. Proceedings of Interspeech 2010, 2010.
9. Man Mohan Sondhi. New methods of pitch extraction. Audio and Electroacoustics, IEEE Transactions on, 16(2):262—266, 1968.
10. Myron J. Ross, Harry L. Shaffer, Asaf Cohen, Richard Freudberg, and Harold J. Manley. Average magnitude difference function pitch extractor. Acoustics, Speech and Signal Processing, IEEE Transactions on, 22(5):353—362, 1974.
11. Na Yang, He Ba, Weiyang Cai, Ilker Demirkol, and Wendi Heinzelman. BaNa: A Noise Resilient Fundamental Frequency Detection Algorithm for Speech and Music. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 22(12):1833–1848, December 2014.
12. Michael Noll. Cepstrum Pitch Determination. The Journal of the Acoustical Society of America, 41(2):293–309, 1967.
13. Jong Wook Kim, Justin Salamon, Peter Li, and Juan Pablo Bello. CREPE: A Convolutional Representation for Pitch Estimation. arXiv:1802.06182 [cs, eess, stat], February 2018. arXiv: 1802.06182.
14. Masanori Morise, Fumiya Yokomori, and Kenji Ozawa. WORLD: A Vocoder-Based High-Quality Speech Synthesis System for Real-Time Applications. IEICE Transactions on Information and Systems, E99.D(7):1877–1884, 2016.
15. Kun Han and DeLiang Wang. Neural Network Based Pitch Tracking in Very Noisy Speech. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 22(12):2158–2168, Decem- ber 2014.
16. Pegah Ghahremani, Bagher BabaAli, Daniel Povey, Korbinian Riedhammer, Jan Trmal, and Sanjeev Khudanpur. A pitch extraction algorithm tuned for automatic speech recognition. In Acoustics, Speech and Signal Processing (ICASSP), 2014 IEEE International Conference on, pages 2494–2498. IEEE, 2014.
17. Lee Ngee Tan and Abeer Alwan. Multi-band summary correlogram-based pitch detection for noisy speech. Speech Communication, 55(7-8):841–856, September 2013.
18. Jesper Kjær Nielsen, Tobias Lindstrøm Jensen, Jesper Rindom Jensen, Mads Græsbøll Christensen, and Søren Holdt Jensen. Fast fundamental frequency estimation: Making a statistically efficient estimator computationally efficient. Signal Processing, 135:188–197, June 2017.
19. Sira Gonzalez and Mike Brookes. PEFAC - A Pitch Estimation Algorithm Robust to High Levels of Noise. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 22(2):518—530, February 2014.
20. Paul Boersma. Accurate short-term analysis of the fundamental frequency and the harmonics-to-noise ratio of a sampled sound. In Proceedings of the institute of phonetic sciences, volume 17, page 97—110. Amsterdam, 1993.
21. David Talkin. A robust algorithm for pitch tracking (RAPT). Speech coding and synthesis, 495:518, 1995.
22. Byung Suk Lee and Daniel PW Ellis. Noise robust pitch tracking by subband autocorrelation classification. In Interspeech, pages 707–710, 2012.
23. Wei Chu and Abeer Alwan. SAFE: a statistical algorithm for F0 estimation for both clean and noisy speech. In INTERSPEECH, pages 2590–2593, 2010.
24. Xuejing Sun. Pitch determination and voice quality analysis using subharmonic-to-harmonic ratio. In Acoustics, Speech, and Signal Processing (ICASSP), 2002 IEEE International Conference on, volume 1, page I—333. IEEE, 2002.
25. Markel. The SIFT algorithm for fundamental frequency estimation. IEEE Transactions on Audio and Electroacoustics, 20(5):367—377, December 1972.
26. Thomas Drugman and Abeer Alwan. Joint Robust Voicing Detection and Pitch Estimation Based on Residual Harmonics. In Interspeech, page 1973—1976, 2011.
27. Hideki Kawahara, Masanori Morise, Toru Takahashi, Ryuichi Nisimura, Toshio Irino, and Hideki Banno. TANDEM-STRAIGHT: A temporally stable power spectral representation for periodic signals and applications to interference-free spectrum, F0, and aperiodicity estimation. In Acous- tics, Speech and Signal Processing, 2008. ICASSP 2008. IEEE International Conference on, pages 3933–3936. IEEE, 2008.
28. Arturo Camacho. SWIPE: A sawtooth waveform inspired pitch estimator for speech and music. PhD thesis, University of Florida, 2007.
29. Kavita Kasi and Stephen A. Zahorian. Yet Another Algorithm for Pitch Tracking. In IEEE International Conference on Acoustics Speech and Signal Processing, pages I–361–I–364, Orlando, FL, USA, May 2002. IEEE.
30. Alain de Cheveigné and Hideki Kawahara. YIN, a fundamental frequency estimator for speech and music. The Journal

The Easiest Way to Collect Data from the Internet Download anything you see on the internet into spreadsheets within a few clicks using our ready-made web crawlers or a few lines of code using our APIs

We have made it as simple as possible to collect data from websites

Easy to Use Crawlers Amazon Product Details and Pricing Scraper Amazon Product Details and Pricing Scraper Get product information, pricing, FBA, best seller rank, and much more from Amazon.

Google Maps Search Results Google Maps Search Results Get details like place name, phone number, address, website, ratings, and open hours from Google Maps or Google Places search results.

Twitter Scraper Twitter Scraper Get tweets, Twitter handle, content, number of replies, number of retweets, and more. All you need to provide is a URL to a profile, hashtag, or an advance search URL from Twitter.

Amazon Product Reviews and Ratings Amazon Product Reviews and Ratings Get customer reviews for any product on Amazon and get details like product name, brand, reviews and ratings, and more from Amazon.

Google Reviews Scraper Google Reviews Scraper Scrape Google reviews and get details like business or location name, address, review, ratings, and more for business and places.

Walmart Product Details & Pricing Walmart Product Details & Pricing Get the product name, pricing, number of ratings, reviews, product images, URL other product-related data from Walmart.

Amazon Search Results Scraper Amazon Search Results Scraper Get product search rank, pricing, availability, best seller rank, and much more from Amazon.

Amazon Best Sellers Amazon Best Sellers Get the bestseller rank, product name, pricing, number of ratings, rating, product images, and more from any Amazon Bestseller List.

Google Search Scraper Google Search Scraper Scrape Google search results and get details like search rank, paid and organic results, knowledge graph, related search results, and more.

Walmart Product Reviews & Ratings Walmart Product Reviews & Ratings Get customer reviews for any product on Walmart.com and get details like product name, brand, reviews, and ratings.

Scrape Emails and Contact Details Scrape Emails and Contact Details Get emails, addresses, contact numbers, social media links from any website.

Walmart Search Results Scraper Walmart Search Results Scraper Get Product details such as pricing, availability, reviews, ratings, and more from Walmart search results and categories.

Glassdoor Job Listings Glassdoor Job Listings Scrape job details such as job title, salary, job description, location, company name, number of reviews, and ratings from Glassdoor.

Indeed Job Listings Indeed Job Listings Scrape job details such as job title, salary, job description, location, company name, number of reviews, and ratings from Indeed.

LinkedIn Jobs Scraper Premium LinkedIn Jobs Scraper Scrape job listings on LinkedIn and extract job details such as job title, job description, location, company name, number of reviews, and more.

Redfin Scraper Premium Redfin Scraper Scrape real estate listings from Redfin. Extract property details such as address, price, mortgage, redfin estimate, broker name and more.

Yelp Business Details Scraper Yelp Business Details Scraper Scrape business details from Yelp such as phone number, address, website, and more from Yelp search and business details page.

Zillow Scraper Premium Zillow Scraper Scrape real estate listings from Zillow. Extract property details such as address, price, Broker, broker name and more.

Amazon product offers and third party sellers Amazon product offers and third party sellers Get product pricing, delivery details, FBA, seller details, and much more from the Amazon offer listing page.

Realtor Scraper Premium Realtor Scraper Scrape real estate listings from Realtor.com. Extract property details such as Address, Price, Area, Broker and more.

Target Product Details & Pricing Target Product Details & Pricing Get product details from search results and category pages such as pricing, availability, rating, reviews, and 20+ data points from Target.

Trulia Scraper Premium Trulia Scraper Scrape real estate listings from Trulia. Extract property details such as Address, Price, Area, Mortgage and more.

Amazon Customer FAQs Amazon Customer FAQs Get FAQs for any product on Amazon and get details like the question, answer, answered user name, and more.

Yellow Pages Scraper Yellow Pages Scraper Get details like business name, phone number, address, website, ratings, and more from Yellow Pages search results.

Introduction The dataset provides detailed information about UK Power Networks' Grid and Primary Sites. It includes key characteristics such as:

Spatial coordinates of each site Year commissioned Asset counts against each site Power transformer count Local authority information Winter and summer demand Transformer ratings

This data is useful for understanding the infrastructure and capacity of the electricity network across its regions.

Methodological Approach

Source: Various internal data domains - geospatial, asset, long term development statement; as well as openly available data from the Ordnance Survey and Office of National Statistics Manipulation: Various data characteristics were combined together using Functional Locations (FLOCs)

Quality Control Statement The data is provided "as is".

Assurance Statement The Open Data team has checked the data against source to ensure data accuracy and consistency. The data domain owners have checked their respective data aspects.

Other Contains data from Office for National Statistics licensed under the Open Government Licence v.3.0. Local Authority District (2022) to Grouped Local Authority District (2022) Lookup for EW - data.gov.uk

Contains Ordnance Survey data Crown copyright and database right [2019-]. Free OS OpenData Map Downloads | Free Vector & Raster Map Data | OS Data Hub

Download dataset information: Metadata (JSON)

Definitions of key terms related to this dataset can be found in the Open Data Portal Glossary: https://ukpowernetworks.opendatasoft.com/pages/glossary/

This repository contains databases of protein domains for use with Foldclass and Merizo-search. We provide databases for all 365 million domains in TED, as well as all classified domains in CATH 4.3.Foldclass and Merizo-search use two formats for databases. The default format uses a PyTorch tensor and a pickled list of Python tuples to store the data. This format is used for the CATH database, which is small enough to fit in memory. For larger-than-memory datasets, such as TED, we use a binary format that is searched using the Faiss library.The CATH database requires approximately 1.4 GB of disk space, whereas the TED database requires about 885 GB. Please ensure you have enough free storage space before downloading. For best search performance with the TED database, the database should be stored on the fastest storage hardware available to you.IMPORTANT:We recommend going in to each folder and downloading the files; if you attempt to download each folder in one go, it will download a zip file which will need to be decompressed. This is particularly an issue if downloading the TED database, as you will need to have roughly twice the storage space needed as compared to downloading the individual files. Our GitHub repository (see Related Materials below) contains a convenience script to download each database; we recommend using that.

This file contains the National Statistics Postcode Lookup (NSPL) for the United Kingdom as at August 2022 in Comma Separated Variable (CSV) and ASCII text (TXT) formats. To download the zip file click the Download button. The NSPL relates both current and terminated postcodes to a range of current statutory geographies via ‘best-fit’ allocation from the 2021 Census Output Areas (national parks and Workplace Zones are exempt from ‘best-fit’ and use ‘exact-fit’ allocations) for England and Wales. Scotland and Northern Ireland has the 2011 Census Output AreasIt supports the production of area based statistics from postcoded data. The NSPL is produced by ONS Geography, who provide geographic support to the Office for National Statistics (ONS) and geographic services used by other organisations. The NSPL is issued quarterly. (File size - 184 MB).

Dataset: The files on your computer.

Crab is a command line tool for Mac and Windows that scans file data into a SQLite database, so you can run SQL queries over it.

e.g. (Win)    C:> crab C:somepathMyProject
or (Mac)    $ crab /some/path/MyProject

You get a CRAB> prompt where you can enter SQL queries on the data, e.g. Count files by extension

SELECT extension, count(*) 
FROM files 
GROUP BY extension;

e.g. List the 5 biggest directories

SELECT parentpath, sum(bytes)/1e9 as GB 
FROM files 
GROUP BY parentpath 
ORDER BY sum(bytes) DESC LIMIT 5;

Crab provides a virtual table, fileslines, which exposes file contents to SQL

e.g. Count TODO and FIXME entries in any .c files, recursively

SELECT fullpath, count(*) FROM fileslines 
WHERE parentpath like '/Users/GN/HL3/%' and extension = '.c'
  and (data like '%TODO%' or data like '%FIXME%')
GROUP BY fullpath;

As well there are functions to run programs or shell commands on any subset of files, or lines within files e.g. (Mac) unzip all the .zip files, recursively

SELECT exec('unzip', '-n', fullpath, '-d', '/Users/johnsmith/Target Dir/') 
FROM files 
WHERE parentpath like '/Users/johnsmith/Source Dir/%' and extension = '.zip';

(Here -n tells unzip not to overwrite anything, and -d specifies target directory)

There is also a function to write query output to file, e.g. (Win) Sort the lines of all the .txt files in a directory and write them to a new file

SELECT writeln('C:UsersSJohnsondictionary2.txt', data) 
FROM fileslines 
WHERE parentpath = 'C:UsersSJohnson' and extension = '.txt'
ORDER BY data;

In place of the interactive prompt you can run queries in batch mode. E.g. Here is a one-liner that returns the full path all the files in the current directory

C:> crab -batch -maxdepth 1 . "SELECT fullpath FROM files"

Crab SQL can also be used in Windows batch files, or Bash scripts, e.g. for ETL processing.

Crab is free for personal use, $5/mo commercial

See more details here (mac): [http://etia.co.uk/][1] or here (win): [http://etia.co.uk/win/about/][2]

An example SQLite database (Mac data) has been uploaded for you to play with. It includes an example files table for the directory tree you get when downloading the Project Gutenberg corpus, which contains 95k directories and 123k files.

To scan your own files, and get access to the virtual tables and support functions you have to use the Crab SQLite shell, available for download from this page (Mac): [http://etia.co.uk/download/][3] or this page (Win): [http://etia.co.uk/win/download/][4]

Content

FILES TABLE

The FILES table contains details of every item scanned, file or directory. All columns are indexed except 'mode'

COLUMNS
 fileid (int) primary key -- files table row number, a unique id for each item
 name (text)        -- item name e.g. 'Hei.ttf'
 bytes (int)        -- item size in bytes e.g. 7502752
 depth (int)        -- how far scan recursed to find the item, starts at 0
 accessed (text)      -- datetime item was accessed
 modified (text)      -- datetime item was modified
 basename (text)      -- item name without path or extension, e.g. 'Hei'
 extension (text)     -- item extension including the dot, e.g. '.ttf'
 type (text)        -- item type, 'f' for file or 'd' for directory
 mode (text)        -- further type info and permissions, e.g. 'drwxr-xr-x'
 parentpath (text)     -- absolute path of directory containing the item, e.g. '/Library/Fonts/'
 fullpath (text) unique  -- parentpath of the item concatenated with its name, e.g. '/Library/Fonts/Hei.ttf'

PATHS
1) parentpath and fullpath don't support abbreviations such as ~ . or .. They're just strings.
2) Directory paths all have a '/' on the end.

FILESLINES TABLE

The FILESLINES table is for querying data content of files. It has line number and data columns, with one row for each line of data in each file scanned by Crab.

This table isn't available in the example dataset, because it's a virtual table and doesn't physically contain data.

COLUMNS
 linenumber (int) -- line number within file, restarts count from 1 at the first line of each file
 data (text)    -- data content of the files, one entry for each line

FILESLINES also duplicates the columns of the FILES table: fileid, name, bytes, depth, accessed, modified, basename, extension, type, mode, parentpath, and fullpath. This way you can restrict which files are searched without having to join tables.

Example Gutenberg data

An example SQLite database (Mac data), database.sqlite, has been uploaded for you to play with. It includes an example files table...

The LIDAR Composite DTM (Digital Terrain Model) is a raster elevation model covering ~99% of England at 1m spatial resolution. The DTM (Digital Terrain Model) is produced from the last or only laser pulse returned to the sensor. We remove surface objects from the Digital Surface Model (DSM), using bespoke algorithms and manual editing of the data, to produce a terrain model of just the surface.

Produced by the Environment Agency in 2022, the DTM is derived from a combination of our Time Stamped archive and National LIDAR Programme surveys, which have been merged and re-sampled to give the best possible coverage. Where repeat surveys have been undertaken the newest, best resolution data is used. Where data was resampled a bilinear interpolation was used before being merged.

The 2022 LIDAR Composite contains surveys undertaken between 6th June 2000 and 2nd April 2022. Please refer to the metadata index catalgoues which show for any location which survey was used in the production of the LIDAR composite.

The data is available to download as GeoTiff rasters in 5km tiles aligned to the OS National grid. The data is presented in metres, referenced to Ordinance Survey Newlyn and using the OSTN’15 transformation method. All individual LIDAR surveys going into the production of the composite had a vertical accuracy of +/-15cm RMSE.

Updates

March 20th, 2025: Added 12089-recommended-screening-list.csv file, which lists unique CR MOFs (ASR, FSR, and ION) from SI, CSD-modified, and CSD-unmodified datasets. While originating from the same source file, the ASR and FSR data differ because ASR structures have the coordinated solvent removed from the open metal sites (OMS), whereas FSR structures retain the solvent coordinated with the OMS.

Web interface for the CoRE MOF SI dataset

https://mof-db.pusan.ac.kr

Full CoRE MOF DB (40,837) = CoRE MOF SI (8,300) + CoRE MOF CSD-modified (20,276) + CoRE MOF CSD-unmodified (12,261)

The dataset is the public version of the CoRE MOF database updated in 2024 ("CoRE MOF SI"), which includes 2,664 computation-ready (CR) and 5,636 not computation-ready (NCR) MOF CIF files (total = 8,300 structures) and precomputed material properties. The dataset includes structures reported up to 12/31/2023 (manuscript acceptance date).

The dataset, based on the structures obtained from the Cambridge Structural Database (CSD) updated in 2024 ("CoRE MOF CSD"), is split into two datasets (unmodified CIFs and modified CIFs).

1. To obtain modified CIFs from CoRE MOF CSD (9,835 CR and 10,441 NCR), please go:

https://www.ccdc.cam.ac.uk/support-and-resources/downloads/

You will need a valid email to log in to the CCDC website to download the dataset for free.

2. To obtain unmodified CIFs from CoRE MOF CSD (4,703 CR and 7,558 NCR), please go:

https://www.ccdc.cam.ac.uk/support-and-resources/downloads/

You will need a CCDC license to obtain the unmodified CIFs.

Precomputed properties: pore limiting diameter (PLD), largest cavity diameter (LCD), pore volume (PV), framework dimensions, accessible surface area, crystal density, topology, open metal site, MOFidv1, MOFidv2, DDEC06 partial atomic charges from PACMAN model, heat capacity, decomposition temperature, probability of solvent removal stability, probability of water stability, hydrophobic classification based on GEMC

Dataset Directory Organization

1. CoREMOF2024DB_SI_20250204.zip: dataset with computation-ready (CR) and not-computation-ready (NCR) classifications come from supporting information

• CR dataset: 2,664
  • ASR (all solvent removed): 1,372
  • FSR (free solvent removed): 1,192
  • Ion (with ions): 100
• NCR dataset: 5,636
  • Both Chen_Manz and mofchecker: 3,692
  • Chen_Manz: 562
  • mofchecker: 1,073
  • occupancy of a single atom is less than 1: 309

2. ASR_data_SI.csv, FSR_data_SI.csv and ION_data_SI.csv: the information of CoRE MOF 2024 ASR, FSR and Ion datasets.

3. NCR_ASR_SI.xlsx, NCR_FSR_SI.xlsx and NCR_ION_SI.xlsx: details of all structures by Chen_Manz and mofchecker for each NCR cases.

4. unmodified_check_for_NCR_SI.xlsx: whether the NCR structures are unmodified according to comparing with the original structure

5. mofid-v2.zip: XYZ files of linkers and metal nodes, errors (which is an "unknown" MOFid)

6. water.zip: GEMC water isotherm data of CR dataset

7. TSA.zip: Single isotherms of 35 MOFs used in TSA; TSA results and adsorption data at different feed conditions

8. ASR_FSR_check.csv: duplicated MOFs from ASR & FSR datasets. We recommend that the researcher remove the structures from this list (one of the columns) for high-throughput screening

9. 12089-recommended-screening-list.csv: lists unique CR MOFs from ASR, FSR, and ION datasets.

All cities with a population > 1000 or seats of adm div (ca 80.000)Sources and ContributionsSources : GeoNames is aggregating over hundred different data sources. Ambassadors : GeoNames Ambassadors help in many countries. Wiki : A wiki allows to view the data and quickly fix error and add missing places. Donations and Sponsoring : Costs for running GeoNames are covered by donations and sponsoring.Enrichment:add country name

This file contains the ONS UPRN Directory (ONSUD) for Great Britain as at January 2022. The ONSUD relates the Unique Property Reference Number (UPRN) for each GB address from AddressBase® Epoch 89 to a range of current statutory administrative, electoral, health and other statistical geographies. The ONSUD is produced by ONS Geography, who provide geographic support to the Office for National Statistics (ONS) and geographic services used by other organisations. The ONSUD is issued every 6 weeks and is designed to complement the Ordnance Survey AddressBase® product. For further technical information about this file, please refer to the User Guide document contained within the downloadable zip file. Please note that this product contains Royal Mail, Gridlink, Ordnance Survey and ONS Intellectual Property Rights. (File Size - 511 MB)