Channel CSV construction and import struggles

OpenGD77CPS
Post Reply
KC1OCY
Posts: 4
Joined: Tue Jan 16, 2024 3:07 pm

Channel CSV construction and import struggles

Post by KC1OCY » Tue Jan 16, 2024 10:25 pm

Hello all:

I'm having problems constructing a CSV list of channels that the CPS will accept. The CSV that I'm trying to use contains a mix of digital and analog channels, and for some reason the CPS imports my digital channels without a problem, but chokes on the analog ones.

When I view the CSV in a text editor, I see that the digital channels that the CPS will import look like this:

Code: Select all

7,Acton U NO1A,Digital,442.35,447.35,12.5,1,,None,MA DMR,None,Off,Off,,,,Master,No,No,No,0,Off,No,No,None,0.128,0.008
The CPS will import analog channels that in a text editor look like this:

Code: Select all

99,Attleboro K1SMH,Analogue,"	147.795","	146.13",25,,,,,,,,None,127.3,Disabled,Master,No,No,No,0,Off,No,No,None,0,0
where there are quotation marks and TABs associated with the RX and TX frequencies. However, if an analog channel does not include the quotes and TABs, like this:

Code: Select all

103,Braintree AE1TH,Analogue,447.5,442.5,25,,,,MA analog,,,,,118.8,Disabled,Master,No,No,No,0,Off,No,No,None,0,0
the CPS throws a "Line ## is invalid and will be skipped" error.

I have 140+ analog repeater and simplex channels that I'd like to include in the my codeplug, and I'm copying the information about them out of other preexisting spreadsheets, so I'd like to build the channel CSV in either Google Sheets or Excel (as opposed to copying each frequency one-by-one into a text file). But no matter what I do, the CSVs that I get out of them look like the line 103 example above (which the CPS refuses), not the line 99 example (which the CPS will accept). Neither Google Sheets nor Excel seem to be able to insert the TAB character that the CPS is looking for (at least, I haven't been able to figure out how to do that). But also, I don't know why the CPS seems to be looking for that character in the analog channels of my CSV and not the digital ones.

Any suggestions as to what I'm missing? (It's probably something obvious that I just don't see.)

Thanks and 73.

VK3KYY
Posts: 7590
Joined: Sat Nov 16, 2019 3:25 am
Location: Melbourne, Australia

Re: Channel CSV construction and import struggles

Post by VK3KYY » Tue Jan 16, 2024 10:50 pm

The CPS will generally only accept the format in which the data was exported, except in a few things like allowing quotes around frequencies.

I have just checked and it can handle if you wrap the frequency in quotes, becuase many spreadsheets seem to do this.

But the CPS can't handle any arbirary additional characters which happen to be in your files

I'd recommend that you build your files using a spreadsheet, but then open the CSV file in a decent text editor, e.g. NotePad++ on Windows, and if necessary turn on the View->Show Symbol -> Show All Characters

and see what extras you have added, and then if its simply things like tabs then do a search and replace and remove them etc



The CPS CSV import works fine as long as you don't give it data it can't handle, and many people have used it for a number of years to import data, because prior to the import function being in the CPS, the same function was available as a stand alone application by Colin G4EML

Colin kindly donated his source code for the CSV import / export to use to use within the CPS

G4EML
Posts: 930
Joined: Sat Nov 16, 2019 10:01 am

Re: Channel CSV construction and import struggles

Post by G4EML » Tue Jan 16, 2024 10:54 pm

The tab character is used to force a spreadsheet program to treat the value as text and not a number. It was intended to prevent channel names (which can contain numbers but are really text) being interpreted as numbers and loosing digits. It is not really needed on frequencies . If a tab is present it should be ignored on import. So it should be optional. At least that is the way it is supposed to work. If anything I would have expected the quotes to be more of a problem.

I will have to check the latest CPS to make sure it still works as intended.

In the mean time if you want to confirm that the tabs are the problem it is easy enough to edit the line in question using a text editor such as notepad. CSV files are just text files after all.

Colin G4EML.

VK3KYY
Posts: 7590
Joined: Sat Nov 16, 2019 3:25 am
Location: Melbourne, Australia

Re: Channel CSV construction and import struggles

Post by VK3KYY » Tue Jan 16, 2024 11:11 pm

The CPS exports with tabs in front of frequencies and imports them OK

In fact I added tabs in front of every field in the Channels.csv file and it still imports without any errors

So whatever is causing problems is not simply quotes or tabs. Perhaps some mix of tabs + quotes or some other characters

Well, possibly quotes around all values may be the problem. But I don't have time to test those various perumtations which are well beyond what the CPS could be reasonably be expected to handle

G4EML
Posts: 930
Joined: Sat Nov 16, 2019 10:01 am

Re: Channel CSV construction and import struggles

Post by G4EML » Tue Jan 16, 2024 11:21 pm

I have just cut and pasted the lines from your post into my channels.csv. Both 99 and 103 imported correctly without errors, so I don't think your problem relates to quotes or tabs.
As Roger says, the CSV import has quite a bit of error checking but it is impossible to predict every possible situation.
Viewing with a text editor is often the way to identify errors in the format.

Colin G4EML

KC1OCY
Posts: 4
Joined: Tue Jan 16, 2024 3:07 pm

Re: Channel CSV construction and import struggles

Post by KC1OCY » Fri Jan 19, 2024 10:48 pm

Thank you all! I'm completely stumped. Currently when I try to import the CSV, I'm getting a "Line 106 is invalid" error, but looking in a text editor (NotePad), I see no structural difference between that line and the line before it. Nor do I see any special characters squirreled away in the file.

I've attached my CSV here. Would someone kindly please take a look at it and, if you see something, let me know what I'm doing wrong?
Attachments
Channels.csv
(29.01 KiB) Downloaded 56 times

G4EML
Posts: 930
Joined: Sat Nov 16, 2019 10:01 am

Re: Channel CSV construction and import struggles

Post by G4EML » Fri Jan 19, 2024 11:26 pm

Two Problems,

You have used NONE instead of None in many lines. The Input Parser is fussy! Use notepad++ to change all NONE to None.

The last Line for WX7 has no channel number (248) at the beginning and no CR LF at the end.

Having corrected these the file loads OK.

Colin G4EML

KC1OCY
Posts: 4
Joined: Tue Jan 16, 2024 3:07 pm

Re: Channel CSV construction and import struggles

Post by KC1OCY » Mon Jan 22, 2024 6:50 pm

That did the trick! Thank you for the feedback, Colin et al, and for introducing me to Notepad++. I've been a Mac user for years and have only recently started getting back into the PC world, so I didn't realize the difference between that and plain old Notepad.

73!
Tom KC1OCY

VK3KYY
Posts: 7590
Joined: Sat Nov 16, 2019 3:25 am
Location: Melbourne, Australia

Re: Channel CSV construction and import struggles

Post by VK3KYY » Mon Jan 22, 2024 8:48 pm

KC1OCY wrote:
Mon Jan 22, 2024 6:50 pm
That did the trick! Thank you for the feedback, Colin et al, and for introducing me to Notepad++. I've been a Mac user for years and have only recently started getting back into the PC world, so I didn't realize the difference between that and plain old Notepad.

73!
Tom KC1OCY
Notepad++ is nothing to do with the old Notepad

It's a truely amazing editor

On PC, Hxd. Hex editor is often handy, as is a program call winmerge.
Winmerge as it's name suggests will visually compare files and let you merge them line by line or block by block of lines etc

Post Reply