added html importer script

README.md

@@ -1,10 +1,15 @@
 # Flame
 
+[![JS Badge](https://img.shields.io/badge/JavaScript-F7DF1E?style=for-the-badge&logo=javascript&logoColor=black)](https://shields.io/)
+[![TS Badge](https://img.shields.io/badge/TypeScript-007ACC?style=for-the-badge&logo=typescript&logoColor=white)](https://shields.io/)
+[![Node Badge](https://img.shields.io/badge/Node.js-43853D?style=for-the-badge&logo=node.js&logoColor=white)](https://shields.io/)
+[![React Badge](https://img.shields.io/badge/React-20232A?style=for-the-badge&logo=react&logoColor=61DAFB)](https://shields.io/)
+
 ![Homescreen screenshot](./.github/_home.png)
 
 ## Description
 
-Flame is self-hosted startpage for your server. Its design is inspired (heavily) by [SUI](https://github.com/jeroenpardon/sui). Flame is very easy to setup and use. With built-in editors it allows you to setup your very own application hub in no time - no file editing necessary.
+Flame is self-hosted startpage for your server. Its design is inspired (heavily) by [SUI](https://github.com/jeroenpardon/sui). Flame is very easy to setup and use. With built-in editors it allows you to setup your very own appliaction hub in no time - no file editing necessary.
 
 ## Technology
 
@@ -37,15 +42,7 @@ npm run dev
 
 ### With Docker (recommended)
 
-[Docker Hub link](https://hub.docker.com/r/pawelmalak/flame)
-
-```sh
-docker pull pawelmalak/flame:latest
-
-# for ARM architecture (e.g. RaspberryPi)
-docker pull pawelmalak/flame:multiarch
-```
-
+[Docker Hub](https://hub.docker.com/r/pawelmalak/flame)
 
 #### Building images
 
@@ -99,13 +96,14 @@ Follow instructions from wiki: [Installation without Docker](https://github.com/
 
 - Applications
   - Create, update, delete and organize applications using GUI
-  - Pin your favourite apps to the homescreen
+  - Pin your favourite apps to homescreen
 
 ![Homescreen screenshot](./.github/_apps.png)
 
 - Bookmarks
   - Create, update, delete and organize bookmarks and categories using GUI
-  - Pin your favourite categories to the homescreen
+  - Pin your favourite categories to homescreen
+  - Import HTML bookmarks (experimental)
 
 ![Homescreen screenshot](./.github/_bookmarks.png)
 
@@ -114,12 +112,24 @@ Follow instructions from wiki: [Installation without Docker](https://github.com/
   - Get current temperature, cloud coverage and weather status with animated icons
 
 - Themes
-  - Customize your page by choosing from 15 color themes
+  - Customize your page by choosing from 12 color themes
 
 ![Homescreen screenshot](./.github/_themes.png)
 
 ## Usage
 
+### Import HTML Bookmarks (Experimental)
+```shell
+
+pip3 install Pillow, beautifulsoup4
+
+cd flame/client/utils/dev
+python3 bookmarks_importer.py --bookmarks <path to bookmarks.html> --data <path to flame data folder>
+```
+
+
+
+
 ### Search bar
 
 #### Searching
@@ -128,7 +138,23 @@ To use search bar you need to type your search query with selected prefix. For e
 
 > You can change where to open search results (same/new tab) in the settings
 
-For list of supported search engines, shortcuts and more about searching functionality visit [project wiki](https://github.com/pawelmalak/flame/wiki/Search-bar).
+#### Supported search engines
+
+| Name       | Prefix | Search URL                          |
+| ---------- | ------ | ----------------------------------- |
+| Disroot    | /ds    | http://search.disroot.org/search?q= |
+| DuckDuckGo | /d     | https://duckduckgo.com/?q=          |
+| Google     | /g     | https://www.google.com/search?q=    |
+
+#### Supported services
+
+| Name               | Prefix | Search URL                                    |
+| ------------------ | ------ | --------------------------------------------- |
+| IMDb               | /im    | https://www.imdb.com/find?q=                  |
+| Reddit             | /r     | https://www.reddit.com/search?q=              |
+| Spotify            | /sp    | https://open.spotify.com/search/              |
+| The Movie Database | /mv    | https://www.themoviedb.org/search?query=      |
+| Youtube            | /yt    | https://www.youtube.com/results?search_query= |
 
 ### Setting up weather module
 
@@ -146,13 +172,13 @@ labels:
   - flame.type=application # "app" works too
   - flame.name=My container
   - flame.url=https://example.com
-  - flame.icon=icon-name # optional, default is "docker"
+  - flame.icon=icon-name # Optional, default is "docker"
 # - flame.icon=custom to make changes in app. ie: custom icon upload
 ```
 
-> "Use Docker API" option must be enabled for this to work. You can find it in Settings > Other > Docker section
+And you must have activated the Docker sync option in the settings panel.
 
-You can also set up different apps in the same label adding `;` between each one.
+You can set up different apps in the same label adding `;` between each one.
 
 ```yml
 labels:
@@ -195,11 +221,13 @@ metadata:
   - flame.pawelmalak/type=application # "app" works too
   - flame.pawelmalak/name=My container
   - flame.pawelmalak/url=https://example.com
-  - flame.pawelmalak/icon=icon-name # optional, default is "kubernetes"
+  - flame.pawelmalak/icon=icon-name # Optional, default is "kubernetes"
 ```
 
-> "Use Kubernetes Ingress API" option must be enabled for this to work. You can find it in Settings > Other > Kubernetes section
+And you must have activated the Kubernetes sync option in the settings panel.
 
-### Custom CSS and themes
+### Custom CSS
 
-See project wiki for [Custom CSS](https://github.com/pawelmalak/flame/wiki/Custom-CSS) and [Custom theme with CSS](https://github.com/pawelmalak/flame/wiki/Custom-theme-with-CSS).
+> This is an experimental feature. Its behaviour might change in the future.
+>
+> Follow instructions from wiki: [Custom CSS](https://github.com/pawelmalak/flame/wiki/Custom-CSS)

client/utils/dev/bookmarks_importer.py

@@ -0,0 +1,167 @@
+import sqlite3
+from bs4 import BeautifulSoup
+from PIL import Image, UnidentifiedImageError
+from io import BytesIO
+import re
+import base64
+from datetime import datetime, timezone
+import os
+import argparse
+
+
+"""
+Imports html bookmarks file into Flame.
+Tested only on Firefox html exports so far.
+
+Usage:
+python3 bookmarks_importer.py --bookmarks <path to bookmarks file> --data <path to flame data dir>
+
+"""
+
+parser = argparse.ArgumentParser()
+parser.add_argument('--bookmarks', type=str, required=True)
+parser.add_argument('--data', type=str, required=True)
+args = parser.parse_args()
+
+bookmarks_path = args.bookmarks
+data_path      = args.data
+created        = datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3] + datetime.now().astimezone().strftime(" %z")
+updated        = created
+if data_path[-1] != '/':
+	data_path = data_path + '/'
+
+
+
+
+def Base64toPNG(codec, name):
+
+	"""
+	Convert base64 encoded image to png file
+	Reference: https://github.com/python-pillow/Pillow/issues/3400#issuecomment-428104239
+
+		Parameters:
+			codec (str): icon in html bookmark format.e.g. 'data:image/png;base64,<image encoding>'
+			name (str): name for export file
+
+		Returns:
+			icon_name(str): name of png output E.g. 1636473849374--mybookmark.png
+			None: if image not produced successfully
+
+	"""
+
+	try:
+		unix_t     = str(int(datetime.now(tz=timezone.utc).timestamp() * 1000))
+		icon_name  = unix_t + '--' + re.sub(r'\W+', '', name).lower() + '.png'
+		image_path = data_path + 'uploads/' + icon_name
+		if os.path.exists(image_path):
+			return image_path
+		base64_data = re.sub('^data:image/.+;base64,', '', codec)
+		byte_data   = base64.b64decode(base64_data)
+		image_data  = BytesIO(byte_data)
+		img         = Image.open(image_data)
+		img.save(image_path, "PNG")
+		return icon_name
+	except UnidentifiedImageError:
+		return None
+
+
+
+
+def FlameBookmarkParser(bookmarks_path):
+
+	"""
+	Parses HTML bookmarks file
+	Reference: https://stackoverflow.com/questions/68621107/extracting-bookmarks-and-folder-hierarchy-from-google-chrome-with-beautifulsoup
+
+		Parameters:
+			bookmarks_path (str): path to bookmarks.html
+
+		Returns:
+			None
+
+	"""
+
+	soup = BeautifulSoup()
+	with open(bookmarks_path) as f:
+        	soup = BeautifulSoup(f.read(), 'lxml')
+
+	dt = soup.find_all('dt')
+	folder_name =''
+	for i in dt:
+		n = i.find_next()
+		if n.name == 'h3':
+			folder_name = n.text
+			continue
+		else:
+			url          = n.get("href")
+			website_name = n.text
+			icon         = n.get("icon")
+			if icon != None:
+				icon_name = Base64toPNG(icon, website_name)
+			cat_id = AddFlameCategory(folder_name)
+			AddFlameBookmark(website_name, url, cat_id, icon_name)
+
+
+
+
+def AddFlameCategory(cat_name):
+	"""
+        Parses HTML bookmarks file
+
+		Parameters:
+			cat_name (str): category name
+
+		Returns:
+			cat_id (int): primary key id of cat_name
+
+        """
+
+
+
+	con       = sqlite3.connect(data_path + 'db.sqlite')
+	cur       = con.cursor()
+	count_sql = ("SELECT count(*) FROM categories WHERE name = ?;")
+	cur.execute(count_sql, [cat_name])
+	count = int(cur.fetchall()[0][0])
+	if count > 0:
+		getid_sql = ("SELECT id FROM categories WHERE name = ?;")
+		cur.execute(getid_sql, [cat_name])
+		cat_id = int(cur.fetchall()[0][0])
+		return cat_id
+
+	is_pinned = 1
+
+	insert_sql = "INSERT OR IGNORE INTO categories(name, isPinned, createdAt, updatedAt) VALUES (?, ?, ?, ?);"
+	cur.execute(insert_sql, (cat_name, is_pinned, created, updated))
+	con.commit()
+
+	getid_sql = ("SELECT id FROM categories WHERE name = ?;")
+	cur.execute(getid_sql, [cat_name])
+	cat_id = int(cur.fetchall()[0][0])
+	return cat_id
+
+
+
+
+def AddFlameBookmark(website_name, url, cat_id, icon_name):
+	con = sqlite3.connect(data_path + 'db.sqlite')
+	cur = con.cursor()
+	if icon_name == None:
+		insert_sql = "INSERT OR IGNORE INTO bookmarks(name, url, categoryId, createdAt, updatedAt) VALUES (?, ?, ?, ?, ?);"
+		cur.execute(insert_sql, (website_name, url, cat_id, created, updated))
+		con.commit()
+	else:
+		insert_sql = "INSERT OR IGNORE INTO bookmarks(name, url, categoryId, icon, createdAt, updatedAt) VALUES (?, ?, ?, ?, ?, ?);"
+		cur.execute(insert_sql, (website_name, url, cat_id, icon_name, created, updated))
+		con.commit()
+
+
+
+
+
+
+
+
+if __name__ == "__main__":
+	FlameBookmarkParser(bookmarks_path)
+