data

所属分类:数据结构
开发工具:Clojure
文件大小:0KB
下载次数:0
上传日期:2023-05-26 20:04:35
上 传 者sh-1993
说明:  从Clojure数据结构到CSV读取器写入器
(CSV reader writer to from Clojure data structures)

文件列表:
CONTRIBUTING.md (478, 2023-08-30)
LICENSE (11516, 2023-08-30)
deps.edn (30, 2023-08-30)
pom.xml (1130, 2023-08-30)
src/ (0, 2023-08-30)
src/main/ (0, 2023-08-30)
src/main/clojure/ (0, 2023-08-30)
src/main/clojure/clojure/ (0, 2023-08-30)
src/main/clojure/clojure/data/ (0, 2023-08-30)
src/main/clojure/clojure/data/csv.clj (4906, 2023-08-30)
src/test/ (0, 2023-08-30)
src/test/clojure/ (0, 2023-08-30)
src/test/clojure/clojure/ (0, 2023-08-30)
src/test/clojure/clojure/data/ (0, 2023-08-30)
src/test/clojure/clojure/data/csv_test.clj (2685, 2023-08-30)

clojure.data.csv ======================================== [*API documentation*](https://clojure.github.io/data.csv/) CSV reader/writer to/from Clojure data structures. Follows the [RFC4180](https://tools.ietf.org/html/rfc4180) specification but is more relaxed. Releases and Dependency Information ======================================== This project follows the version scheme MAJOR.MINOR.PATCH where each component provides some relative indication of the size of the change, but does not follow semantic versioning. In general, all changes endeavor to be non-breaking (by moving to new names rather than by breaking existing names). Latest stable release: 1.0.1 * [All Released Versions](https://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.clojure%22%20AND%20a%3A%22data.csv%22) * [Development Snapshot Versions](https://oss.sonatype.org/index.html#nexus-search;gav~org.clojure~data.csv~~~) [CLI/`deps.edn`](https://clojure.org/reference/deps_and_cli) dependency information: ```clojure org.clojure/data.csv {:mvn/version "1.0.1"} ``` [Leiningen](https://github.com/technomancy/leiningen) dependency information: ```clojure [org.clojure/data.csv "1.0.1"] ``` [Maven](http://maven.apache.org/) dependency information: ```xml org.clojure data.csv 1.0.1 ``` Example Usage ======================================== ```clojure (require '[clojure.data.csv :as csv] '[clojure.java.io :as io]) (with-open [reader (io/reader "in-file.csv")] (doall (csv/read-csv reader))) (with-open [writer (io/writer "out-file.csv")] (csv/write-csv writer [["abc" "def"] ["ghi" "jkl"]])) ``` Refer to the [API documentation](https://clojure.github.io/data.csv/) for additional information. ## Working with data.csv This library is meant to remain small and focus on nothing but correctly parsing csv files. The following sections describes how to effectively use data.csv as a building block in larger programs as well as some hints on how to solve common problems. ### Laziness When parsing a csv file with data.csv the result is a lazy sequence of vectors of strings. With some care, laziness makes it possible to process very large csv files without excessive memory use. Here's an example of a program that copies one csv file to another but drops the first and last columns: ```clojure (defn copy-csv [from to] (with-open [reader (io/reader from) writer (io/writer to)] (->> (csv/read-csv reader) (map #(rest (butlast %))) (csv/write-csv writer)))) ``` This function will work even if the csv file is larger than would fit in memory because all the steps are lazy. There are a few things to look out for when dealing with lazy sequences. Especially with data.csv where the sequence is often created via a `clojure.java.io/reader` that could already be closed when the lazy sequence is consumed. For example ```clojure (defn read-column [filename column-index] (with-open [reader (io/reader filename)] (let [data (csv/read-csv reader)] (map #(nth % column-index) data)))) (defn sum-second-column [filename] (->> (read-column filename 1) (drop 1) ;; Drop header column (map #(Double/parseDouble %)) (reduce + 0))) ``` This program will throw the exception "`java.io.Exception`: Stream Closed". The reason is that both `read-csv` and `map` are lazy, so `read-column` will immeditaly return a sequence without actually reading any bytes from the file. The reading (and parsing) will happen when data is needed by the calling code (`reduce` in this case). By the time `reduce` tries to add the first value `with-open` will already have closed the `io/reader` and the exception is thrown. There are two solutions to this problem: 1. Move the opening/closing of the reader up the callstack to the point where the content is consumed: ```clojure (defn read-column [reader column-index] (let [data (csv/read-csv reader)] (map #(nth % column-index) data))) (defn sum-second-column [filename] (with-open [reader (io/reader filename)] (->> (read-column reader 1) (drop 1) (map #(Double/parseDouble %)) (reduce + 0)))) ``` 2. Don't return a lazy sequence ```clojure (defn read-column [filename column-index] (with-open [reader (io/reader filename)] (let [data (csv/read-csv reader)] ;; mapv is not lazy, so the csv data will be consumed at this point (mapv #(nth % column-index) data)))) (defn sum-second-column [filename] (->> (read-column filename 1) (drop 1) (map #(Double/parseDouble %)) (reduce + 0))) ``` Which approach to choose depends on the application. If the csv file isn't huge the second approach will often work well. If you want to be careful not to read the csv file into memory the first approach is preferable. ### Parsing into maps Data.csv parses lines of a csv file into a vector of strings. This is often not the desired output where you might want the result to be a sequence of maps instead, such as ```text foo,bar,baz A,1,x B,2,y C,3,z ``` ```clojure ({:foo "A" :bar "1" :baz "x"} {:foo "B" :bar "2" :baz "y"} {:foo "C" :bar "3" :baz "z"}) ``` One fairly elegant way to achieve this is the expression ```clojure (defn csv-data->maps [csv-data] (map zipmap (->> (first csv-data) ;; First row is the header (map keyword) ;; Drop if you want string keys instead repeat) (rest csv-data))) (csv-data->maps (csv/read-csv reader)) ``` This function is lazy so all the options described in the previous section are still valid. Now that the data is in a nice format it's easy to do any desired post-processing: ```clojure (->> (csv/read-csv reader) csv-data->maps (map (fn [csv-record] (update csv-record :bar #(Long/parseLong %))))) ({:foo "A" :bar 1 :baz "x"} {:foo "B" :bar 2 :baz "y"} {:foo "C" :bar 3 :baz "z"}) ``` ### Byte Order Mark A [byte order mark (BOM)](https://en.wikipedia.org/wiki/Byte_order_mark) is a byte sequence that appears as the first couple of bytes in some CSV files (and other text files). Data.csv will not automatically remove these extra bytes so they can accidentally be interpreted as part of the first cells characters. If you want to avoid this you can either try to manually detect it by looking at the first byte(s) and calling `(.skip reader 1)` before you pass the reader to read-csv. Another option is to create the reader in such a way that the BOM will be automatically removed. One way to achieve this is to use [`org.apache.commons.io.input/BOMInputStream`](https://commons.apache.org/proper/commons-io/javadocs/api-release/org/apache/commons/io/input/BOMInputStream.html): ```clojure (with-open [reader (-> "data.csv" io/input-stream BOMInputStream. io/reader)] (doall (csv/read-csv reader))) ``` Developer Information ======================================== * [GitHub project](https://github.com/clojure/data.csv) * [Bug Tracker](https://clojure.atlassian.net/browse/DCSV) * [Continuous Integration](https://github.com/clojure/data.csv/actions/workflows/test.yml) Change Log ==================== * Release 1.0.1 on 2022-04-04 * Lift construction requiring quote out of the quote? function in write-csv * Release 1.0.0 on 2020-02-18 * Release 0.1.4 on 2017-04-05 * [DCSV-16](https://dev.clojure.org/jira/browse/DCSV-16) Resolve some reflection warnings * Release 0.1.3 on 2015-08-10 * [DCSV-4](http://dev.clojure.org/jira/browse/DCSV-4) Allow carriage return by itself as a record separator * Release 0.1.2 on 2012-02-24 * Fixed keyword params for `write-csv` * Release 0.1.1 on 2012-02-14 * Added quote? keyword param to write-csv * Code cleanup * Release 0.1.0 on 2011-08-26 * Initial release. Copyright and License ======================================== Copyright (c) Jonas Enlund, Rich Hickey, and contributors, 2012-2023. All rights reserved. The use and distribution terms for this software are covered by the Eclipse Public License 1.0 (https://opensource.org/license/epl-1-0/) which can be found in the file epl-v10.html at the root of this distribution. By using this software in any fashion, you are agreeing to be bound by the terms of this license. You must not remove this notice, or any other, from this software.

近期下载者

相关文件


收藏者