Ideas for my next freeware project
Ideas for my next freeware project
Hello,
I just finished my first freeware project (CSV Buddy), posted here a few days ago.
I'll make sure I have time to maintain it, following suggestions or bug reports I may receive. But it is time for me to think about my next freeware project.
Is this the right forum to bounce such ideas I have or should I do it in the Chit-Chat area?
I just finished my first freeware project (CSV Buddy), posted here a few days ago.
I'll make sure I have time to maintain it, following suggestions or bug reports I may receive. But it is time for me to think about my next freeware project.
Is this the right forum to bounce such ideas I have or should I do it in the Chit-Chat area?
Re: Ideas for my next freeware project
How about a podcast suite?
Features could include RSS Podcast downloads with rules (both auto and manual), Audio cutting, Streaming (& stream recording), Playing MP3s. (Have not spotted built in audio cutting in any podcatcher)
Examples of these features in other apps.
Re: Podcast downloads
See Roberts Podcatcher This app did a never-ending download when auto download was allowed not fun but a common problem since no podcast software I have tested has a set of rules to go with auto-download.
Gpodder This app works great most of the time but can hang and is hard to use from some mp3 players also Windows has a hard time copying it.
Ziepod This app downloads to a specific path and duplicates the downloaded file/s have not found a relative path setting.
Re: Audio cutting
See AudioBook Cutter FE This app cuts audio files at regular intervals during silence. This is great for longer audio files some podcasts are 2-3 hours.
Re: Streaming
See Screamer Radio This app is great streamer and stream recorder, and it is easy to add streams.
Re: Playing MP3s
CoolPlayer This app can minimize down to a sliver and can be placed over the title-bar of a web-browser. (using the pre-installed skins)
Features could include RSS Podcast downloads with rules (both auto and manual), Audio cutting, Streaming (& stream recording), Playing MP3s. (Have not spotted built in audio cutting in any podcatcher)
Examples of these features in other apps.
Re: Podcast downloads
See Roberts Podcatcher This app did a never-ending download when auto download was allowed not fun but a common problem since no podcast software I have tested has a set of rules to go with auto-download.
Gpodder This app works great most of the time but can hang and is hard to use from some mp3 players also Windows has a hard time copying it.
Ziepod This app downloads to a specific path and duplicates the downloaded file/s have not found a relative path setting.
Re: Audio cutting
See AudioBook Cutter FE This app cuts audio files at regular intervals during silence. This is great for longer audio files some podcasts are 2-3 hours.
Re: Streaming
See Screamer Radio This app is great streamer and stream recorder, and it is easy to add streams.
Re: Playing MP3s
CoolPlayer This app can minimize down to a sliver and can be placed over the title-bar of a web-browser. (using the pre-installed skins)
Re: Ideas for my next freeware project
Thanks for your suggestion Donald. But, I already have some ideas that are more in continuity with my previous project.
In fact, I have two spin-off ideas from my CSV Buddy project. CSV Buddy is a Windows portable app that reads, edits and writes CSV files. A secondary feature of this app is to export the CSV data to various formats including HTML page.
My first idea is to focus the new app on HTML page creation. This "HTML generator" would read CSV files (with the same read options found in CSV Buddy), edit, sort data, etc. The main new feature would be an editor/wizard to support the creation of the HTML template. A template consists of three sections: page header (what precedes the list), the list itself (the rows of data) and the page footer. In the row template, variables are linked to fields in the list and are replaced with the field's content for each row. Other variables could be inserted in the header or footer (page title, source, dates, file name, etc.).
The other idea seems more of a challenge to me. In a nutshell, the app would scrape an HTML page containing a structured list, build a table with the data in this list and save it to a CSV format. This "HTML extractor" wizard would help analyze the HTML source to mark the list beginning and end, and to markup each field in the list. Of course, the licence agreement would state that this app must be used only to extract data legally available to the user.
I know these kinds of tools exist web-based or as part of a database or HTML editors. But I don't know if they exist in the world of portable software? And if there is an interest in it?
Any comment / expression of interest / ideas would be appreciated.
In fact, I have two spin-off ideas from my CSV Buddy project. CSV Buddy is a Windows portable app that reads, edits and writes CSV files. A secondary feature of this app is to export the CSV data to various formats including HTML page.
My first idea is to focus the new app on HTML page creation. This "HTML generator" would read CSV files (with the same read options found in CSV Buddy), edit, sort data, etc. The main new feature would be an editor/wizard to support the creation of the HTML template. A template consists of three sections: page header (what precedes the list), the list itself (the rows of data) and the page footer. In the row template, variables are linked to fields in the list and are replaced with the field's content for each row. Other variables could be inserted in the header or footer (page title, source, dates, file name, etc.).
The other idea seems more of a challenge to me. In a nutshell, the app would scrape an HTML page containing a structured list, build a table with the data in this list and save it to a CSV format. This "HTML extractor" wizard would help analyze the HTML source to mark the list beginning and end, and to markup each field in the list. Of course, the licence agreement would state that this app must be used only to extract data legally available to the user.
I know these kinds of tools exist web-based or as part of a database or HTML editors. But I don't know if they exist in the world of portable software? And if there is an interest in it?
Any comment / expression of interest / ideas would be appreciated.
Re: Ideas for my next freeware project
My suggestion would be a program that can convert between CSV, JSON and XML. They are all intended to make it easier for the output from one program to be readable by another, but I keep finding that what one program outputs is one that the next program I want to use cannot read.
Re: Ideas for my next freeware project
@ Craunch Have you tried this?
http://shancarter.github.io/mr-data-converter/
http://shancarter.github.io/mr-data-converter/
Re: Ideas for my next freeware project
@tproli thanks for that link, but it only converts from CSV to JSON etc, but not from JSON etc to CSV.
At the moment I am more interested in getting things into CSV which can be a bit messy. I could write this myself, but if I did it would almost certainly be written in Java as a command line utility which would exclude it from this site on two counts.
There is also http://www.json2csv.com/, but I dislike relying on internet sites to remain available if I need to do something. I much prefer having my software on my computer.
At the moment I am more interested in getting things into CSV which can be a bit messy. I could write this myself, but if I did it would almost certainly be written in Java as a command line utility which would exclude it from this site on two counts.
There is also http://www.json2csv.com/, but I dislike relying on internet sites to remain available if I need to do something. I much prefer having my software on my computer.
Re: Ideas for my next freeware project
Hi there, JnLlnd. Nice of you to ask fellow TPFCers for input on your projects.
JnLlnd wrote:The other idea seems more of a challenge to me. In a nutshell, the app would scrape an HTML page containing a structured list, build a table with the data in this list and save it to a CSV format. This "HTML extractor" wizard would help analyze the HTML source to mark the list beginning and end, and to markup each field in the list. Of course, the licence agreement would state that this app must be used only to extract data legally available to the user.
I know these kinds of tools exist web-based or as part of a database or HTML editors. But I don't know if they exist in the world of portable software? And if there is an interest in it?
Any comment / expression of interest / ideas would be appreciated.
- I personally like this idea. One of the gaps I have felt in the world of portability is for a nice and easy Web scraper. Programs like Cintanotes come really close -- but when they turn commercial, the need to insure client satisfaction tends to get in the way of the KISS philosophy. I have used both the MS MHT and Mozilla's MAFF format for personal web archiving (in this respect, MixBrowser, a commercial Trident based front-end, once looked like it was advancing in the right direction; as does Mayakron's freeware CHreader -- also at Softpedia), but every ready made solution falls short one way or the other: no metadata (mainly, URL and date), or no HTML, or no export/import of collected data, etc...
Craunch wrote:My suggestion would be a program that can convert between CSV, JSON and XML. They are all intended to make it easier for the output from one program to be readable by another, but I keep finding that what one program outputs is one that the next program I want to use cannot read.
- This surely looks like an interesting feature set, especially if it could improve on the editing facilities of a tiny little pearl like XML Marker (sadly not free, not portable and not developed anymore). Free Virtual Tree Navigator has some ideas of its own to offer here, too, although it's close to unusable for the laymen...
Re: Ideas for my next freeware project
@Craunch: Have you tried the XML export feature in CSV Buddy? It does convert your CSV data to the XML format. You must ensure that field names and field data comply with the rules of XML syntax.
This simple example, where each record has two fields named "Field1" and "Field2", shows the XML output format:
<?xml version='1.0'?>
<XMLExport>
<Record>
<Field1>Value Row 1 Col 1</Field1>
<Field2>Value Row 1 Col 2</Field1>
</Record>
<Record>
<Field1>Value Row 2 Col 1</Field1>
<Field2>Value Row 2 Col 2</Field1>
</Record>
</XMLExport>
Not enough time to read/reply to other comments. I'll be back later. Have a good day!
This simple example, where each record has two fields named "Field1" and "Field2", shows the XML output format:
<?xml version='1.0'?>
<XMLExport>
<Record>
<Field1>Value Row 1 Col 1</Field1>
<Field2>Value Row 1 Col 2</Field1>
</Record>
<Record>
<Field1>Value Row 2 Col 1</Field1>
<Field2>Value Row 2 Col 2</Field1>
</Record>
</XMLExport>
Not enough time to read/reply to other comments. I'll be back later. Have a good day!
Re: Ideas for my next freeware project
@JnLlnd CSV Buddy is a nice piece of work, and I must admit to not having spotted the XML export feature which could come in useful. But I need to get JSON or XML data into CSV in the first place. This shouldn't be particularly difficult, but it does get a bit messy if the JSON/XML data has any sort of nested structure. A good example of this would be a Firefox bookmark backup file, which has a tree structure in JSON.
Keep up the good work, see you later.
Keep up the good work, see you later.
Re: Ideas for my next freeware project
@Craunch: I never used JSON before but adding "Export to JSON" to CSV Buddy's export tab seems quite easy (a few hours of work). But the opposite (JSON -> CSV), as you mention, is more complex if not impossible in the case of hierarchical data, CSV being so flat...
Converting HTML, XML or JSON to CSV are all variations of the same challenge: parsing semi-freeform or hierarchical data. It means analyzing the source to get field names, finding record delimiters, and start/end of each fields. Interesting enough. If I go in this direction, I would write a text-based parser/wizard (ie you would work in source code) to stay in the simple side of things (but this would target users with some knowledge of the source language). There are very sophisticated web-based application that support WYSIWYG parsing (like [url="http://www.automationanywhere.com/solut ... ataExt.htm"]Automation Everywhere]) but they cost much $$$.
@Midas: Thank you for your suggestions and references. I'll look at the examples you mention and come back to you after I make my mind on them. But a question: are you interested in web scraping for any kind of content or specifically for structured data (that could be saved as CSV, for example)?
No interest yet for an HTML generator from s CSV table?
Converting HTML, XML or JSON to CSV are all variations of the same challenge: parsing semi-freeform or hierarchical data. It means analyzing the source to get field names, finding record delimiters, and start/end of each fields. Interesting enough. If I go in this direction, I would write a text-based parser/wizard (ie you would work in source code) to stay in the simple side of things (but this would target users with some knowledge of the source language). There are very sophisticated web-based application that support WYSIWYG parsing (like [url="http://www.automationanywhere.com/solut ... ataExt.htm"]Automation Everywhere]) but they cost much $$$.
@Midas: Thank you for your suggestions and references. I'll look at the examples you mention and come back to you after I make my mind on them. But a question: are you interested in web scraping for any kind of content or specifically for structured data (that could be saved as CSV, for example)?
No interest yet for an HTML generator from s CSV table?
Re: Ideas for my next freeware project
Hierarchical JSON --> CSV is not impossible - you just need a good flattening algorithm
Let me give an example based on something you use every day, the Windows filing system.
Currently the directory that I downloaded CSVBuddy to contains 1 files and a subdirectory with 2 files:
If I give their full pathnames they are:
Hierarchically this can be represented as, and this is similar to how the data would be represented in JSON:
In tabular form this could appear as:
or as CSV:
I hope this has given you something to think about
Let me give an example based on something you use every day, the Windows filing system.
Currently the directory that I downloaded CSVBuddy to contains 1 files and a subdirectory with 2 files:
Code: Select all
csvbuddy.zip
CSVBuddy-0_8_1-BETA\CSVBuddy-0_8_1-BETA-32-bits.exe
CSVBuddy-0_8_1-BETA\CSVBuddy-0_8_1-BETA-64-bits.exe
Code: Select all
E:\Craunch\downloads\CSVBuddy\
E:\Craunch\downloads\CSVBuddy\CSVBuddy-0_8_1-BETA\CSVBuddy-0_8_1-BETA-32-bits.exe
E:\Craunch\downloads\CSVBuddy\CSVBuddy-0_8_1-BETA\CSVBuddy-0_8_1-BETA-64-bits.exe
Code: Select all
Drive = E:
Dir1 = Craunch
Dir2 = Downloads
Dir3 = CSVBuddy
Filename = csvbuddy.zip
Dir4 = CSVBuddy-0_8_1-BETA
Filename = CSVBuddy-0_8_1-BETA-32-bits.exe
Filename = CSVBuddy-0_8_1-BETA-64-bits.exe
Code: Select all
Drive Dir1 Dir2 Dir3 Dir4 Filename
E: Craunch Downloads CSVBuddy csvbuddy.zip
E: Craunch Downloads CSVBuddy CSVBuddy-0_8_1-BETA CSVBuddy-0_8_1-BETA-32-bits.exe
E: Craunch Downloads CSVBuddy CSVBuddy-0_8_1-BETA CSVBuddy-0_8_1-BETA-64-bits.exe
Code: Select all
Drive,Dir1,Dir2,Dir3,Dir4,Filename
E:,Craunch,Downloads,CSVBuddy,,csvbuddy.zip
E:,Craunch,Downloads,CSVBuddy,CSVBuddy-0_8_1-BETA,CSVBuddy-0_8_1-BETA-32-bits.exe
E:,Craunch,Downloads,CSVBuddy,CSVBuddy-0_8_1-BETA,CSVBuddy-0_8_1-BETA-64-bits.exe
Re: Ideas for my next freeware project
JnLlnd wrote:@Midas: [...] But a question: are you interested in web scraping for any kind of content or specifically for structured data (that could be saved as CSV, for example)?
- Short answer is: general content -- but including structured data.
Longer rationale: my main concern has always been keeping data generally accessible and dodging application/format lock-in; I just can't stand the notion that anything a user has worked on at some point in time might be rendered irretrievable/inaccessible because software developers thought that their program functionality could/should override user data.
So interoperability, normalization/standardization, refresh and migration (or entry/exit) paths are always one of my main concerns when evaluating software. CSV, as a long-standing open data standard, plays a part there, as do most text based (human readable) data formats. It's also the main drive of my interest in portable software.
By way of example, I still have email messages in my system(s) that are over ten years old...
Re: Ideas for my next freeware project
If I develop an app, it would focus on structured data. But I understand ang agree with your concerns about data accessibility.Midas wrote:
- Short answer is: general content -- but including structured data.
Good point! Notes takenCraunch wrote:Hierarchical JSON --> CSV is not impossible - you just need a good flattening algorithm
...
I hope this has given you something to think about
Re: Ideas for my next freeware project
Good to see a new developer amongst us.
Re: Ideas for my next freeware project
Thank you guiness. I took a look at your site. I'll give a try to your installer for sure.