html.parser — Simple HTML and XHTML parser — Python …
Source code: Lib/html/
This module defines a class HTMLParser which serves as the basis for
parsing text files formatted in HTML (HyperText Mark-up Language) and XHTML.
class (*, convert_charrefs=True)¶
Create a parser instance able to parse invalid markup.
If convert_charrefs is True (the default), all character
references (except the ones in script/style elements) are
automatically converted to the corresponding Unicode characters.
An HTMLParser instance is fed HTML data and calls handler methods
when start tags, end tags, text, comments, and other markup elements are
encountered. The user should subclass HTMLParser and override its
methods to implement the desired behavior.
This parser does not check that end tags match start tags or call the end-tag
handler for elements which are closed implicitly by closing an outer element.
Changed in version 3. 4: convert_charrefs keyword argument added.
Changed in version 3. 5: The default value for argument convert_charrefs is now True.
Example HTML Parser Application¶
As a basic example, below is a simple HTML parser that uses the
HTMLParser class to print out start tags, end tags, and data
as they are encountered:
from import HTMLParser
class MyHTMLParser(HTMLParser):
def handle_starttag(self, tag, attrs):
print(“Encountered a start tag:”, tag)
def handle_endtag(self, tag):
print(“Encountered an end tag:”, tag)
def handle_data(self, data):
print(“Encountered some data:”, data)
parser = MyHTMLParser()
(‘
‘
Parse me!
‘)
The output will then be:
Encountered a start tag: html
Encountered a start tag: head
Encountered a start tag: title
Encountered some data: Test
Encountered an end tag: title
Encountered an end tag: head
Encountered a start tag: body
Encountered a start tag: h1
Encountered some data: Parse me!
Encountered an end tag: h1
Encountered an end tag: body
Encountered an end tag: html
HTMLParser Methods¶
HTMLParser instances have the following methods:
(data)¶
Feed some text to the parser. It is processed insofar as it consists of
complete elements; incomplete data is buffered until more data is fed or
close() is called. data must be str.
()¶
Force processing of all buffered data as if it were followed by an end-of-file
mark. This method may be redefined by a derived class to define additional
processing at the end of the input, but the redefined version should always call
the HTMLParser base class method close().
Reset the instance. Loses all unprocessed data. This is called implicitly at
instantiation time.
Return current line number and offset.
t_starttag_text()¶
Return the text of the most recently opened start tag. This should not normally
be needed for structured processing, but may be useful in dealing with HTML “as
deployed” or for re-generating input with minimal changes (whitespace between
attributes can be preserved, etc. ).
The following methods are called when data or markup elements are encountered
and they are meant to be overridden in a subclass. The base class
implementations do nothing (except for handle_startendtag()):
HTMLParser. handle_starttag(tag, attrs)¶
This method is called to handle the start of a tag (e. g.
The tag argument is the name of the tag converted to lower case. The attrs
argument is a list of (name, value) pairs containing the attributes found
inside the tag’s <> brackets. The name will be translated to lower case,
and quotes in the value have been removed, and character and entity references
have been replaced.
For instance, for the tag ‘)
Decl: DOCTYPE HTML PUBLIC “-//W3C//DTD HTML 4. 01//EN” ”
Parsing an element with a few attributes and a title:
>>> (‘
Start tag: img
attr: (‘src’, ”)
attr: (‘alt’, ‘The Python logo’)
>>>
>>> (‘
Python
‘)
Start tag: h1
Data: Python
End tag: h1
The content of script and style elements is returned as is, without
further parsing:
>>> (‘
‘)
Start tag: style
attr: (‘type’, ‘text/css’)
Data: #python { color: green}
End tag: style
>>> (‘‘)
Start tag: script
attr: (‘type’, ‘text/javascript’)
Data: alert(“hello! “);
End tag: script
Parsing comments:
>>> (‘‘… ‘IE-specific content‘)
Comment: a comment
Comment: [if IE 9]>IE-specific content‘):
>>> (‘>>>’)
Named ent: >
Num ent: >
Feeding incomplete chunks to feed() works, but
handle_data() might be called more than once
(unless convert_charrefs is set to True):
>>> for chunk in [‘
Start tag: span
Data: buff
Data: ered
Data: text
End tag: span
Parsing invalid HTML (e. unquoted attributes) also works:
>>> (‘
Start tag: p
Start tag: a
attr: (‘class’, ‘link’)
attr: (‘href’, ‘#main’)
Data: tag soup
End tag: p
End tag: a
Creating and Viewing HTML Files with Python | Programming Historian
Contents
Lesson Goals
Files Needed For This Lesson
Creating HTML with Python
“Hello World” in HTML using Python
Using Python to Control Firefox
Mac Instructions
Windows Instructions
Suggested Readings
Code Syncing
This lesson uses Python to create and view an HTML file. If you write
programs that output HTML, you can use any browser to look at your
results. This is especially convenient if your program is automatically
creating hyperlinks or graphic entities like charts and diagrams.
Here you will learn how to create HTML files with Python scripts, and
how to use Python to automatically open an HTML file in Firefox.
If you do not have these files from the previous lesson, you can
download programming-historian-5, a zip file from the previous lesson.
At this point, we’ve started to learn how to use Python to download
online sources and extract information from them automatically. Remember
that our ultimate goal is to incorporate programming seamlessly into our
research practice. In keeping with this goal, in this lesson and the
next, we will learn how to output data back as HTML. This has a few
advantages. First, by storing the information on our hard drive as an
HTML file we can open it with Firefox and use Zotero to index and
annotate it later. Second, there are a wide range of visualization
options for HTML which we can draw on later.
If you have not done the W3 Schools HTML tutorial yet, take a few
minutes to do it before continuing. We’re going to be creating an HTML
document using Python, so you will have to know what an HTML document
is!
One of the more powerful ideas in computer science is that a file that
seems to contain code from one perspective can be seen as data from
another. It is possible, in other words, to write programs that
manipulate other programs. What we’re going to do next is create an HTML
file that says “Hello World! ” using Python. We will do this by storing
HTML tags in a multiline Python string and saving the contents to a new
file. This file will be saved with an extension rather than a
extension.
Typically an HTML file begins with a doctype declaration. You saw
this when you wrote an HTML “Hello World” program in an earlier lesson.
To make reading our code easier, we will omit the doctype in this
example. Recall a multi-line string is created by enclosing the text in
three quotation marks (see below).
#
f = open(”, ‘w’)
message = “””
Hello World!
“””
(message)
()
Save the above program as and execute it. Use File ->
Open in your chosen text editor to open to verify that
your program actually created the file. The content should look like
this:
HTML Source Generated by Python Program
Now go to your Firefox browser and choose File -> New Tab, go to the
tab, and choose File -> Open File. Select You
should now be able to see your message in the browser. Take a moment to
think about this: you now have the ability to write a program which can
automatically create a webpage. There is no reason why you could not
write a program to automatically create a whole website if you wanted
to.
We automatically created an HTML file, but then we had to leave our
editor and go to Firefox to open the file in a new tab. Wouldn’t it be
cool to have our Python program include that final step? Type or copy
the code below and save it as When you execute it, it
should create your HTML file and then automatically open it in a new tab
in Firefox. Sweet!
Mac users will have to specify to the precise location of the
file on their computer. To do this, locate the programming-historian
folder you created to do these tutorials, right-click it and select “Get
Info”.
You can then cut and paste the file location listed after “Where:” and
make sure you include a trailing slash (/) to let the computer know you
want something inside the directory (rather than the directory itself).
import webbrowser
#Change path to reflect file location
filename = ‘file/Users/username/Desktop/programming-historian/’ + ”
_new_tab(filename)
If you’re getting a “File not found” error you haven’t changed the
filename path correctly.
_new_tab(”)
***
Not only have you written a Python program that can write simple HTML,
but you’ve now controlled your Firefox browser using Python. In the next
lesson, we turn to outputting the data that we have collected as an HTML
file.
Lutz, Learning Python
Re-read and review Chs. 1-17
To follow along with future lessons it is important that you have the
right files and programs in your “programming-historian” directory. At
the end of each lesson in the series you can download the “programming-historian” zip
file to make sure you have the correct code. If you are following along
with the Mac / Linux version you may have to open the file and
change “file/Users/username/Desktop/programming-historian/” to the
path to the directory on your own computer.
zip sync
7.5. HTMLTags – generate HTML in Python – Karrigell 3.1.1
7. 5. 1 Overview
The HTMLTags module defines a class for all the valid HTML tags, written in
uppercase letters. To create a piece of HTML, the general syntax is:
t = TAG(content, key1=val1, key2=val2,… )
so that print t results in:
For instance:
print A(‘bar’, href=”foo”)
==>
bar
Attributes with the same name as Python keywords (class,
type) must be capitalized:
print DIV(‘bar’, Class=”title”)
To generate HTML attributes without value, give them the value
True:
print OPTION(‘foo’, SELECTED=True, value=5)