How to extract text with respect to the heading?

[Laxmi530]

Hai,
Thank you for providing a beautiful library.
Actually, I am trying to extract portion of PDF with respect to the heading like in the sample pdf file as a input we give the heading 'At glance 1' so as output need the text from 'The team' to ' penalty shootout.'.

so can someone please guide me how to do this.

Thank you in advance.

Sample_PDF_file.pdf

[pubpub-zz]

PDF is mainly an electronic print format. The Headings notion does not exist. Sometimes some extra features like bookmarks (not available in your sample) exist but those are only providing jumps to absolution position.
The only solution may be a complex use a visitor function to analyse font and text.
https://pypdf.readthedocs.io/en/stable/user/extract-text.html#using-a-visitor

[Laxmi530]

@MartinThoma you are correct, but for machine learning we need data and time.

As of my knowledge we can extract the headings on the basics of text font size, and we can select the text from a heading position to the next heading position. It looks bit tricky. Need your further opinion.
So can someone please guide me how can I get the text font size and text position.

[MartinThoma]

https://pypdf.readthedocs.io/en/stable/user/extract-text.html#using-a-visitor

[J0hn3ch]

Hi @MartinThoma have you find a good example on How to use visitor? I tried to do something follow the documentation, but due to my little knowledge about text matrix, it's very difficult for me doing something that work with visitor_text.

I have the same problem about extract text with heading and bold text

Thank you for your time and help!

[stefan6419846]

What exactly is your issue with the simple visitor examples inside the docs? Yes, there are lots of values which require a deeper knowledge, but as mentioned in the docs, all text extraction is hard and usually requires you to dig deeper into some stuff and the PDF specification.

Another simple example for a visitor could be something like this:

from collections import defaultdict
from pypdf import PdfReader

reader = PdfReader("Sample_PDF_file.pdf")
page = reader.pages[0]


lines = defaultdict(list)
size_by_line = defaultdict(list)


def visitor_body(text, cm, tm, font_dict, font_size):
    y = tm[5]
    lines[y].append(text)
    size_by_line[y].append(font_size)


page.extract_text(visitor_text=visitor_body)

for y in sorted(lines, reverse=True):
    line = lines[y]
    size = size_by_line[y]
    print(''.join(line).strip('\n'), size)

This will extract the text lines in the correct order and display the corresponding reported font sizes. You will observe that even for this simple example document, lines might already be split across multiple visitor calls and that some font sizes "bleed" into the wrong line. (Did I already mention that good text extraction is hard?) I assume that all snippets of a line are returned in the correct reading order, thus ignoring the x coordinates. Correctly ordering the snippets of a line is left as a rather easy task to the reader and should be rather obvious by reading the above docs.

As already mentioned by Martin above, finding a solution for a specific PDF file might be easy. But as soon as you are going to expand your solution to thousands of real-world examples, you will probably stumble about all sorts of strange and unexpected edge cases and have to dig deeper into the PDF stuff anyway if you want to understand them.

[J0hn3ch]

@stefan6419846 Thank you for detailed answer. The documentation is good, i keep only poor attention to this due to deadline. I work with legal documents and I agree with your words about lof of edge cases in working with PDFs.