CSV Format Requirements for File Connectors

This article outlines the required CSV formatting standards to ensure proper schema detection, data preview, and ingestion through File Connectors in the Data Catalog.

CSV files are processed using strict parsing rules. Structural inconsistencies such as incorrect row breaks, inconsistent delimiters, or improper quoting may prevent schema detection and cause the Data Tab to display “No Data Exist” even when the file contains data.

Following standard CSV formatting ensures that files are processed correctly, data is displayed accurately, and file connectors work consistently.

Supported CSV Structure Requirements

To ensure successful processing, CSV files must comply with the standards below.

1. Row Consistency

• Each record must occupy exactly one physical line.

• All rows must contain the same number of columns.

• Rows must not break across multiple lines unless properly escaped according to standard format rules.

2. Quoting Rules

• Either all fields requiring quotes must be consistently quoted, or none.

• Mixed quoting within the same column is not supported.

• If double quotes are used, they must follow standard escaping rules.

Correct Format Example:

FileName,FileType,FileSize
report1,1200, Operations
report2,3400, Department

If quoting is required:

"John","Finance"
"Rao","Operations"

3. Embedded Newlines Not Supported

CSV files must not contain embedded newline characters inside quoted fields.

Not Supported:

"John,
Operations,
Human Resources"

When a single field spans multiple physical lines, the parser cannot reliably determine where one record ends and the next begins.

• Schema auto-detection failure

• All rows treated as malformed

• Data ignored during read execution

4. Multi-Value Columns

If a column contains multiple values, they must be:

• Stored in a single line

• Separated by a supported delimiter such as a comma or a pipe

• Without line breaks

Supported Format:

Finance,Operations,Human Resources

5. Delimiter Consistency

• The same delimiter must be used across the entire file.

• Comma is the recommended delimiter.

• Do not mix different delimiters, such as a comma and a semicolon, within the same file.

6. Header Row Requirement

• The first row must contain column headers.

• Headers must not contain line breaks.

• Duplicate column names are not supported.

Common Cause of “No Data Exist” in the Data Tab

The Data Tab may display “No Data Exist” when:

• Embedded newline characters exist inside quoted values.

• Column counts vary between rows.

• Quoting patterns are inconsistent.

• Multiple delimiters are used within the same file.

• The schema cannot be inferred due to structural ambiguity.

• The analytical engine rejects malformed rows during strict parsing.

Even if data appears visible in spreadsheet software, structural inconsistencies may cause ingestion failure during strict parsing.

Validation Recommendation Before Upload

Before ingesting CSV files into the Data Catalog:

1. Ensure each record occupies one physical line.

2. Confirm that all rows contain the same number of columns.

3. Verify consistent use of delimiters across the file.

4. Ensure quoting is applied consistently where required.

5. Confirm that no embedded newline characters exist inside fields.

6. Validate the file against standard CSV formatting rules before upload.

All CSV files must follow the RFC 4180 standard formatting to ensure compatibility across analytical engines and file connectors.

---

Copyright © 2026, OvalEdge LLC, Peachtree Corners GA USA