What is KindleEar?

KindleEar is a free webapp that running on Google App Engine, it aggregates feeds and deliver the generated docs to your Kindle periodically.

Features:

Benefits:

Dows this application require its own server?

This application is hosted on Google App Engine, it's fine if you don't own a server.

How to deploy my own delivery server?

The formal way:

  1. Download the latest version of KindleEar on github and then extract it to somewhere, for example C:\KindleEar (we'll use it in the following steps).
  2. You need a GAE account, please visit the Google AppEngine page. Use your gmail account to login, click on "Create Application", select "others" in the account activation SMS page, enter your cellphone number with your country code (+nn), click "Send". You will receive an activation message on the phone in a while, after activating your account, click "Create Application" again, in the "Application Identifier" field, enter a name of your applications (e.g. Kindleear123). The remaining options can be kept as default , click on the "Create Application" button to create a GAE application.
  3. Install Python 2.7.x. Download the latest 2.7.x source release and then install it. Do not install a Python 3.x version, because GAE does not support it now.
  4. Install GAE SDK, which will be used to upload a GAE program. In section on this page, click "Google App Engine SDK for Python", choose your platform, click to download the corresponding version, then install, all the way "next" without words.
  5. Go to the previously downloaded KindleEar directory (e.g. C:\KindleEar), use a text editor to open app.yaml and module-worker.yaml file, modify the word 'kindleear' in the first line to your GAE application name (e.g. kindleear123), then save and exit.
  6. Modify variables 'SRC_EMAIL' and 'DOMAIN' and 'TIMEZONE' in config.py in same directory of app.yaml. You can modify other variables if you want and understand what it means.Using the default values are recommended.
  7. Start upload KindleEar now. You can use Google App Engine Launcher GUI or command line. Because Launcher is not stable, we'll use the command line in Windows.
  8. After successfully uploading KindleEar, use your browser to open the site: http://kindleear123.appspot.com (example only, kindleear123 should be changed to the name of your GAE application). If everything goes fine, then go to the last step.
  9. Add your gmail account to your kindle email whitelist. You should know how to do it already.
  10. Then what? Enjoy your own delivery service now!

The easy way(recommended):

  1. Download KindleEar and extract it, get a GAE account, create an application.
  2. Download uploader. (for windows user only)
  3. Extract 'uploader' to a directory, move the folder of KindleEar into the folder of 'uploader' and rename it to kindleear, then double-click 'uploader.bat' to upload.

Why do I not receive the deliveries?

There are many possible reasons:

  1. You haven't enabled the "Enable deliver custom rss" option, this option is at the bottom of the setting page.
  2. You haven't added your gmail address to Amazon's 'Approved Personal Document E-mail List'.
  3. The content of the rss link isn't a valid XML format.

What is full text rss?

Full text rss means that it contains all the contents of the articles in the rss, while summary rss contains only brief descriptions of the articles. You can check it by opening the link of the rss.

Can full text rss be processed as summary rss? What about the reverse?

Full text rss can be treated as summary rss. In this case, program will ignore the content in the rss xml file and fetch full text from the original link. It costs much time which will result the decreasing of supported rss quantities. If summary rss is processed as full text rss, articles sent to your kindle would still have brief descriptions only.

How to deal with weekly subscription?

Some rss are updated weekly. For this type of rss, you can touch a new python file in the 'books' directory with variable 'deliver_days' setted to a day(e.g. deliver_days=['Friday']).

Can KindleEar deliver news to my kindle more than once a day?

The answer is yes, follow the next steps to make it come true:

  1. Add a python file to the directory of 'books'.
  2. Set 'deliver_times' using a list in the new python file.
    For example: deliver_times=[6,14,22] #6:00,14:00,22:00 deliver three times a day
  3. Optional: set 'oldest_article' to the number of interval seconds between two deliveries.
    For example: oldest_article=28800 #8*60*60

How to use the url filter?

Url filter is not for beginners, you need to know regular expression and python. Urls in the list would not be downloaded for saving time.

What can I do if the gfw of China blocked some rss?

Change the link to use https protocol.

How to put the cover or the masthead images in the same directory when I add a customize book?

Set the variable mastheadfile and coverfile to a path, for example:

How to deal with the SSL error problem when I upload KindleEar?

Some friends in China would encounter "urllib2.URLError: <urlopen error [Errno 8] _ssl.c:504: EOF occurred in violation of protocol>". To solve this problem, try to add a new line "74.125.229.174 appengine.google.com" to the hosts file.

Why some articles have garbled characters?

For custom rss and the built-in books without encoding specified, program detects encoding automatically by using http header, html meta or the chardet module of python. Howerver, the result is not 100% accurate. If the encoding detecting is wrong, the article may be garbled. Because this situation is very rare, KindleEar only supplies an not very elegant solution: manually change the chardet-detect encoding saved in the database. Steps as follows:

  1. Login appid.appspot.com by using account 'admin'(assuming your application name is appid).
  2. Visit 'https://appid.appspot.com/dbviewer'. Check the encoding of the website whose articles are garbled in the UrlEncoding section. If the encoding is wrong, take down its ID.
  3. Now visit 'https://appid.appspot.com/dbviewer?action=modurlenc&id=dbid&feedenc=fenc&pageenc=penc'
  4. Or you can change them in 'Datastore viewer' in the GAE backend.
  5. You can get the correct encoding by reading source code of a webpage.
  6. If you cannot find the website in page 'dbviewer', it means KindleEar has never detected the encoding of any page of this website, http header or html meta is used only. In this case, you can set the variable 'ALWAYS_CHAR_DETECT' in 'config.py' to True.
  7. If the article still has garbled characters after setting 'ALWAYS_CHAR_DETECT' to True, it means that encoding detected by chardet is wrong and doesn't match with the http header or the html meta. The final attempt is to set 'TRUST_ENCODING_IN_HEADER_OR_META' to True.
  8. With all the solutions tested and the problem still exists, please add the rss to a python file in the folder 'books' with encoding specified.

What can I do if I forget the password?

If you are a non-administrator user, and you forget the password, you can ask the administrator to change your password. If the administrator password was forgot, you can login appengine.google.com, select "Datastore Admin" on the left, delete all KeUser data, all the account information will be deleted. And then re-use admin/admin to login in your application . Did not find "Datastore Admin"? You can activate it in the "Application Settings".
If you know python, I'm sure that you can find other solutions.

How to use mail service xxx@appid.appspotmail.com?

You'll have uncountable mail addresses like xxx@appid.appspotmail.com as long as xxx is valid ascii string after you have deployed KindleEar(appid is your application id).

There are some logs like 'fetch rss failed(450)' in the gae backend, what's the meaning of them?

The number in parentheses is an error code. Besides the standard error codes, hereafter shows errorcodes defined by KindleEar.

Why I can't lookup words in the delivery books after I chosen 'Merge books into one'? 

With the option 'Merge books in one' checked, KindleEar will set the metadata of the book as Custom Rss. To fix this problem, you have to set the language attribute in the 'Custom Rss Setting' section manually.

How to deal with those websites that need login before I can view the articles?

If you want to crawl a website that needs login, you have to add a python file to the folder 'books'. Refer to sample books if you don't know how to write it.

  1. needs_subscription : Set to True
  2. login_url : The URL for login page
  3. form_4_login : Optional, if not set, KindleEar will guess which form in the page is used to login. Set to an integer indicate the sequence number (where 0 is the first) of the form in the page. Set to a string indicate the name of the form, you can use CSS selector: '#' for id and '.' for class, string only will match any of it(name/id/class).
  4. Then login to KindleEar. In the 'Feeds' page, a new 'Login info' button will appear on the right side of the book which needs_subscription setted to True in the 'Subscripted' section. Click on it and then fill in your account name and password.

Note1: currently we don't support those sites that need run javascript to login or need to input a captcha.

Note2: for some special site, you may have to overwirte the 'login' function which inherits from the 'BaseFeedBook' class.

Note3: the password will be stored encrypted on GAE, the key for encryption is a random string for each account of KindleEar, but the key is also stored on GAE, so it's not absolutely safe, please keep this in mind.

Can I delete/hide books in Chinese?

If you are not Chinese, you can delete some *.py files in directory 'books'.
'__init__.py' and 'base.py' are required, please don't remove them.

What's the meaning of the "Separate" option for the built-in books?

If you choose this option, the book you chosen will be sent to kindle separately even if the option "Merge books into one" is chosen.

I'm sorry for my poor English, please feel free to open an issue on github when you get puzzled, and it will be appreciated if you can help me improve this documentation.