Retrieving Minimal Information

[euler]

A new script has been made available to members to retrieve minimal information relating to the problems and account details. It is necessary to be logged in (an active session), but once this is done you can use the following three calls:

http://projecteuler.net/minimal=problems
This will return a new line for each active problem in the database containing the following information (delimted by ##).
id, description/title, date_published (timestamp), date_last_updated (timestamp), solved_by (number of members), solved_flag (0=no / 1=yes), answer (if solved_flag=1 otherwise empty)

http://projecteuler.net/minimal=problems;csv
This provides the same information as above except it will force the browser to open/save as a comma separated file (delimited by commas).

http://projecteuler.net/minimal=X (where X is the problem id)
This will return the raw problem content data in HTML.

http://projecteuler.net/minimal=profile
This will return details for the current account in a minimal format (delimited by ##).
username, alias, coutnry, language, problems_solved (total number), level, solved_string
[The solved string is a "binary string" equal in length to the highest active problem and contains either 0 or 1 at a position corresponding to the problem id]

http://projecteuler.net/minimal=new
Returns a list of the release dates of any new problems (one per line if more than one and delimited by ##).
id, release_date (timestamp)
[If no new problems are currently scheduled then it will display the message, "No new problems currently scheduled"]

[sivakd]

euler, thanks for these. If I may suggest, why not generate csv or tsv files instead of html output with ## separators? csv/tsv files can be easily opened with Excel for analysis.

[euler]

I've added support to retrieve the minimal information for problems as a csv file. See the main post for how to access this feature.

I used ## as a delimiter because not all languages have native support for comma separated files and as the description/title in the problems can contain commas it is not as simple as just using a string to array function with a comma delimiter. Also none of the minimal options specified text/html output, but because I was not specifying a header the output was being best interpreted as text/html by default. I've now changed the other minimal outputs to use text/plain.

[sivakd]

euler, thanks for the csv support. I think there is no easy way to provide a pure text (stripping out the html tags and entities) header for csv. Also, in addition to the solved flag, providing the rank if available would be good.

[euler]

What is the rank?

[sivakd]

Sorry, I meant the position in the fastest solvers table.

[SoboLAN]

These pages are very useful for developers, thank you euler.

There is however a problem: introducing the CSV page is not a good idea because there are some problem titles that contain commas (","). CSV parsers can be easily confused by this. Using "##" is much better.

EDIT: I saw now that you already found this problem, you can ignore this post.

[SoboLAN]

Except for the problems 1, 143, 395 and 396, all problems have the same value in the "last_update" field: 18th September 2012. Why is this ? (I suspect why, I want to see if I'm right )

[Homez]

Hello to all folks!

I'm a Project Euler common solver (level 7) but I started thinking sometime ago of writing some framework to do keep some things more simple. This could be a JAVA SE application maintained in Eclipse. I could, for example, see the same list of prolems with their short descriptions, my solution status and the number of people having solved this problem. This now can all be taken from Project Euler site thanks yo yours scripts, euler! From me I could add there my actual solution times, so I could be eager to improve them. The solutions could run inside this app, so solution times could be maintained automatically in some config file or database like SQLite. It could use also some sorting and/or filtering so I could easily find problems solved by many people but which I haven't solved already by myself (I do it all now manually because I beleive that the problem I solve next is always among my unsolved problems that were solved by a larger quantity of members).

I could write all this just for my personal usage, but I could also share it to people that would like that (of course, without a single solution or common algorithm).

But I wonder: how can I login programmatically in order to get rid of these scripts? I know that I have to deal with session cookies in order to stay logged but I can't even login from my Java starting app

I saw this nice sample: http://ubuntuincident.wordpress.com/2011/11/08/download-login-protected-pages-with-python-using-mechanize-and-splinter-part-3/ Here some library browsers are used with Python. But I'm not ready to switch to Python, now I'm a Java guy

[Homez]

I can procceed! I've just added "login=true" to POST params - and now I see the list of problems, page 1. Now I can try to ged rid of session.

[Homez]

And I can't I try to set cookie that I got after posting a login form, for the next query. But I always get About page, being logged off. Can anyone help?

[Sjums]

Hey euler, I'm all new (problem 8) but already hooked! Thanks for a nice riddle

A simple request for the minimal data is; would it be possible to strip the HTML and serve it as plain text? I'm specifically thinking of the problem detail page:-)

Regards, sjums

[euler]

The problem content depends on HTML as it may contain images, superscript/subscript, or data contained in tables. If this formatting is removed then it would be impossible to interpret the problem correctly; for example, consider x = 3² without superscript: x = 32.

[SoboLAN]

Hi,

It makes sense to need an active session when going to http://projecteuler.net/minimal=problems or http://projecteuler.net/minimal=profile. The output seen in these cases depends on the user currently logged in.

But can you please remove this requirement for http://projecteuler.net/minimal=X (raw HTML of a problem) ? The output is always independent of the session, so it makes no sense in requiring it.

Programmatically logging in or automatically including cookies in the software written to retrieve the HTML is actually more difficult than I thought. It's a problem we shouldn't have...

[lucaswerkmeister]

can you please remove this requirement for http://projecteuler.net/minimal=X (raw HTML of a problem) ? The output is always independent of the session, so it makes no sense in requiring it.

Yes, that would be super cool... I just deployed a broken update to my plugin because of that.