mardi 26 janvier 2021

How can I copy the content of a website to a Markdown file with Python?

I want to write a Python function that takes an URL as an argument and outputs a Markdown file with the content of the webpage. The images embedded in the website should be download and appropriately referenced in the Markdown file.

I wrote this code

import requests
import html2text

% The URL
link = "https://www.some.website"
f = requests.get(link)

% URL content to plain text (HTML)
textHtml = f.text

% HTML text to MD text
h = html2text.HTML2Text()
textMd = h.handle(textHtml)

% MD text is written to file
text_file = open("output.md", "w")
text_file.write(textMd)
text_file.close()

I think it does the job in downloading the text and formatting it into a Markdown file but I don't know how to download images and add references in the Markdown file to the local image files.

How can I do this?

Thanks in advance!




Aucun commentaire:

Enregistrer un commentaire