How to extract text with respect to the heading?

[Laxmi530]

Hai,
Thank you for providing a beautiful library.
Actually, I am trying to extract portion of text with respect to the heading like in the sample pdf file we select the heading 'At glance 1' so as output need the text from 'The team' to ' penalty shootout.'. I tried some technique, but it is not working also tried to extract the headings so on the basis of position of headings we can process.
so can someone please guide how to do this.

Thank you in advance.

Sample_PDF_file.pdf

[jsvine]

I would suggest an approach like this:

1. Use page.extract_words(extra_attrs=["fontname", "size"]) to extract the words on the page, plus information about the character fonts and sizes.
2. Use the information from step 1 to identify the positions of the headers.
3. Use the information from step 2 plus page.crop(...).extract_text() to get the text between each header. (The ... will be a bounding box that you construct based on each consecutive pair of header positions.)

[danltw]

Thanks @jsvine , managed to do it with "doctop"

[Laxmi530]

Hai @danltw,
Sorry for the late response.
What I did actually 1st count the font name and font size 2nd drop the most repeated font name and font size and consider the remaining font name and font size on that basics you will get the heading names. Post getting the heading names you can apply indexing. This is not a more generic way to extract the text with respect to the heading.

Can you please share your approach?

Thank you.

[danltw]

@Laxmi530 My code is an adaptation from this article https://towardsdatascience.com/extracting-headers-and-paragraphs-from-pdf-using-pymupdf-676e8421c467

However, I couldn't continue with PyMuPDF as it wasn't permissive in licensing, thus changed to pdfplumber instead.

The tl;dr approach is as follows (as adapted from above article):

• First iteration through the doc to extract the fontname and size attributes, doing a count of the unique combinations of fontname_fontsize.
• Then do a reverse sort so that the highest used combination of fontname_fontsize aka first element in the sorted list will be taken as the default paragraph font size. With this information, I can safely tag this combination as a paragraph, in my case, I chose to prepend html tags - <p> paragraphs, <h1> headers, <s2> subscripts. This is just my way of tagging, you may choose something else that makes sense and can be easily identified.
• Current iteration of my code will do 2 steps... first would be assuming that headers fonts would be larger than the presumed paragraph font, and subscripts smaller. This is a simple comparator of fontsize from the fontname_fontsize identifiers from above.
• However, I found that my sample PDFs have headers that are the same size as the paragraph fontsize. Hence, I currently assume bold as headers for now, and do a cleanup when I am joining back the words into lines downstream

Sorry in advance, kinda bad in explaining

[Laxmi530]

Thanks @danltw you explained well.
will try your solution.

Thanks for your explanation.

[jaideep11061982]

@danltw would you like to provide the link for your approach ?

[bansalsahab]

glanc

hey,
did you get the code?