[16.0][MIG] account_statement_import_online_ofx: Migration to 16.0

account_statement_import_online_ofx/README.rst

@@ -0,0 +1,105 @@
+===========================
+Online Bank Statements: OFX
+===========================
+
+.. 
+   !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
+   !! This file is generated by oca-gen-addon-readme !!
+   !! changes will be overwritten.                   !!
+   !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
+   !! source digest: sha256:e6ebad00d6c39c92584c6ee76b5deae328742e8a81229971ce4ca37d2f1e4cb1
+   !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
+
+.. |badge1| image:: https://img.shields.io/badge/maturity-Beta-yellow.png
+    :target: https://odoo-community.org/page/development-status
+    :alt: Beta
+.. |badge2| image:: https://img.shields.io/badge/licence-AGPL--3-blue.png
+    :target: http://www.gnu.org/licenses/agpl-3.0-standalone.html
+    :alt: License: AGPL-3
+.. |badge3| image:: https://img.shields.io/badge/github-OCA%2Fbank--statement--import-lightgray.png?logo=github
+    :target: https://github.com/OCA/bank-statement-import/tree/16.0/account_statement_import_online_ofx
+    :alt: OCA/bank-statement-import
+.. |badge4| image:: https://img.shields.io/badge/weblate-Translate%20me-F47D42.png
+    :target: https://translation.odoo-community.org/projects/bank-statement-import-16-0/bank-statement-import-16-0-account_statement_import_online_ofx
+    :alt: Translate me on Weblate
+.. |badge5| image:: https://img.shields.io/badge/runboat-Try%20me-875A7B.png
+    :target: https://runboat.odoo-community.org/builds?repo=OCA/bank-statement-import&target_branch=16.0
+    :alt: Try me on Runboat
+
+|badge1| |badge2| |badge3| |badge4| |badge5|
+
+This module provides online bank statements from Open Financial Exchange (OFX) institutions.
+You can set-up your own provider, or import a list of supported providers.
+https://ofxhome.com/ is used as a data source, currently over 300 institutions  are supported.
+
+**Table of contents**
+
+.. contents::
+   :local:
+
+Configuration
+=============
+
+To configure online bank statements provider:
+
+#. Go to *Invoicing > Configuration > Online Bank Statement Providers*
+#. Create a provider and configure provider-specific settings.
+
+If you want to allow empty bank statements to be created every time the
+information is pulled, you can check the option "Allow empty statements"
+at the provider configuration level.
+
+**NOTE**: To access these features, user needs to belong to
+*Show Full Accounting Features* group.
+
+Usage
+=====
+
+To pull historical bank statements:
+
+#. Go to *Invoicing > Configuration > Journals*
+#. Select specific bank accounts
+#. Launch *Actions > Online Bank Statements Pull Wizard*
+#. Configure date interval and click *Pull*
+
+Bug Tracker
+===========
+
+Bugs are tracked on `GitHub Issues <https://github.com/OCA/bank-statement-import/issues>`_.
+In case of trouble, please check there if your issue has already been reported.
+If you spotted it first, help us to smash it by providing a detailed and welcomed
+`feedback <https://github.com/OCA/bank-statement-import/issues/new?body=module:%20account_statement_import_online_ofx%0Aversion:%2016.0%0A%0A**Steps%20to%20reproduce**%0A-%20...%0A%0A**Current%20behavior**%0A%0A**Expected%20behavior**>`_.
+
+Do not contact contributors directly about support or help with technical issues.
+
+Credits
+=======
+
+Authors
+~~~~~~~
+
+* ForgeFlow
+
+Contributors
+~~~~~~~~~~~~
+
+* `ForgeFlow <https://www.forgeflow.com/>`__
+
+  * Jasmin Solanki <jasmin.solanki@forgeflow.com>
+
+Maintainers
+~~~~~~~~~~~
+
+This module is maintained by the OCA.
+
+.. image:: https://odoo-community.org/logo.png
+   :alt: Odoo Community Association
+   :target: https://odoo-community.org
+
+OCA, or the Odoo Community Association, is a nonprofit organization whose
+mission is to support the collaborative development of Odoo features and
+promote its widespread use.
+
+This module is part of the `OCA/bank-statement-import <https://github.com/OCA/bank-statement-import/tree/16.0/account_statement_import_online_ofx>`_ project on GitHub.
+
+You are welcome to contribute. To learn how please visit https://odoo-community.org/page/Contribute.

account_statement_import_online_ofx/__init__.py

@@ -0,0 +1,4 @@
+# License AGPL-3.0 or later (https://www.gnu.org/licenses/agpl).
+
+from . import models
+from . import wizards

account_statement_import_online_ofx/__manifest__.py

@@ -0,0 +1,22 @@
+# Copyright 2023 ForgeFlow S.L.
+# License AGPL-3.0 or later (https://www.gnu.org/licenses/agpl.html).
+
+{
+    "name": "Online Bank Statements: OFX",
+    "version": "16.0.1.0.0",
+    "author": "ForgeFlow, Odoo Community Association (OCA)",
+    "website": "https://github.com/OCA/bank-statement-import",
+    "license": "AGPL-3",
+    "category": "Accounting",
+    "summary": "Online bank statements for OFX",
+    "depends": [
+        "account_statement_import_online",
+    ],
+    "external_dependencies": {"python": ["ofxtools", "ofxparse"]},
+    "data": [
+        "security/ir.model.access.csv",
+        "views/online_bank_statement_provider.xml",
+        "wizards/online_bank_statement_pull_wizard.xml",
+    ],
+    "installable": True,
+}

account_statement_import_online_ofx/models/__init__.py

@@ -0,0 +1,4 @@
+# License AGPL-3.0 or later (https://www.gnu.org/licenses/agpl).
+
+from . import ofx_institution
+from . import online_bank_statement_provider_ofx

account_statement_import_online_ofx/models/ofx_institution.py

@@ -0,0 +1,14 @@
+# Copyright 2023 ForgeFlow S.L.
+# License AGPL-3.0 or later (https://www.gnu.org/licenses/agpl.html).
+
+
+from odoo import fields, models
+
+
+class OFXInstitution(models.Model):
+    _name = "ofx.institution"
+    _description = "OFX Institution"
+
+    name = fields.Char()
+    nickname = fields.Char()
+    ofxhome_id = fields.Integer()

account_statement_import_online_ofx/models/online_bank_statement_provider_ofx.py

@@ -0,0 +1,187 @@
+# Copyright 2023 ForgeFlow S.L.
+# License AGPL-3.0 or later (https://www.gnu.org/licenses/agpl.html).
+
+import io
+import time
+import xml.etree.ElementTree as ET
+from datetime import datetime
+
+import requests
+from ofxparse import OfxParser
+from ofxtools import ofxhome, utils
+from ofxtools.Client import OFXClient, StmtRq
+
+from odoo import _, api, fields, models
+from odoo.exceptions import UserError
+
+
+class OnlineBankStatementProviderOFX(models.Model):
+    _inherit = "online.bank.statement.provider"
+
+    ofx_institution_line_ids = fields.One2many("ofx.institution.line", "provider_id")
+
+    @api.model
+    def _get_available_services(self):
+        return super()._get_available_services() + [
+            ("OFX", "OFX"),
+        ]
+
+    def get_statements(self, client, password, s1, try_no=1):
+        statements = client.request_statements(password, s1, skip_profile=True)
+        file_data = statements.read()
+        if b"Request unsuccessful. Incapsula incident ID" in file_data and try_no <= 3:
+            time.sleep(3)
+            return self.get_statements(client, password, s1, try_no + 1)
+        ofx = OfxParser.parse(io.BytesIO(file_data))
+        if ofx.status.get("code") != 0:
+            raise UserError(file_data)
+        return ofx
+
+    def _obtain_statement_data(self, date_since, date_until):
+        self.ensure_one()
+        is_scheduled = self.env.context.get("scheduled")
+        # set date_until as today's date if its greter than today's date
+        # to avoid failure of request.
+        today = datetime.now()
+        today = today.replace(hour=0, minute=0, second=0, microsecond=0)
+        if date_until > today:
+            date_until = today
+        if self.service != "OFX":
+            return super()._obtain_statement_data(
+                date_since,
+                date_until,
+            )  # pragma: no cover
+
+        lines = []
+        if is_scheduled:
+            ofx_institution_lines = self.ofx_institution_line_ids
+        else:
+            ofx_institution_lines = self.env["ofx.institution.line"].browse(
+                self._context.get("ofx_institution_ids")
+            )
+        for ofx_institution_line in ofx_institution_lines:
+            try:
+                username = ofx_institution_line.username
+                password = ofx_institution_line.password
+                bankid = ofx_institution_line.bankid
+                ofxhome_id = ofx_institution_line.institution_id.ofxhome_id
+                acctid = ofx_institution_line.account_id
+
+                institute = ofxhome.lookup(ofxhome_id)
+                if institute is None or institute.url is None:
+                    raise UserError(_("OFX Data is not available"))
+                ofxhome_id = institute.id
+                client = OFXClient(
+                    institute.url,
+                    userid=username,
+                    org=institute.org,
+                    fid=institute.fid,
+                    bankid=bankid,
+                )
+                dtstart = date_since.replace(tzinfo=utils.UTC)
+                dtend = date_until.replace(tzinfo=utils.UTC)
+                s1 = StmtRq(
+                    acctid=acctid,
+                    dtstart=dtstart,
+                    dtend=dtend,
+                    accttype="CHECKING",
+                )
+                ofx = self.get_statements(client, password, s1)
+                for account in ofx.accounts:
+                    if not account.statement.transactions:
+                        continue
+                    for transaction in account.statement.transactions:
+                        vals = self._prepare_ofx_transaction_line(transaction)
+                        if vals:
+                            lines.append(vals)
+            except Exception as e:
+                raise UserError(
+                    _("The following problem occurred during import.\n\n %s") % str(e)
+                ) from e
+        return lines, {}
+
+    @api.model
+    def _prepare_ofx_transaction_line(self, transaction):
+        payment_ref = transaction.payee
+        if transaction.checknum:
+            payment_ref += " " + transaction.checknum
+        if transaction.memo:
+            payment_ref += " : " + transaction.memo
+        vals = {
+            "date": transaction.date,
+            "payment_ref": payment_ref,
+            "amount": float(transaction.amount),
+            "unique_import_id": transaction.id,
+        }
+        return vals
+
+    def import_ofx_institutions(self):
+        OfxInstitution = self.env["ofx.institution"]
+        try:
+            with requests.get(
+                "http://www.ofxhome.com/api.php?all=yes", timeout=30
+            ) as f:
+                response = f.text
+            institute_list = {
+                fi.get("id").strip(): fi.get("name").strip()
+                for fi in ET.fromstring(response)
+            }
+        except Exception as e:
+            raise UserError(_(e)) from e
+
+        for ofxhome_id, name in institute_list.items():
+            institute = OfxInstitution.search([("ofxhome_id", "=", ofxhome_id)])
+            vals = {
+                "name": name,
+                "ofxhome_id": ofxhome_id,
+            }
+            if institute:
+                institute.write(vals)
+            else:
+                OfxInstitution.create(vals)
+
+    def _create_or_update_statement(
+        self, data, statement_date_since, statement_date_until
+    ):
+        # deleting blank statement for OFX online import
+        res = super()._create_or_update_statement(
+            data, statement_date_since, statement_date_until
+        )
+        if self.service == "OFX":
+            unfiltered_lines, statement_values = data
+            if not unfiltered_lines:
+                unfiltered_lines = []
+            if not statement_values:
+                statement_values = {}
+            statement_values["name"] = self.make_statement_name(statement_date_since)
+            filtered_lines = self._get_statement_filtered_lines(
+                unfiltered_lines,
+                statement_values,
+                statement_date_since,
+                statement_date_until,
+            )
+            if not filtered_lines:
+                return
+            if filtered_lines:
+                statement_values.update(
+                    {"line_ids": [[0, False, line] for line in filtered_lines]}
+                )
+            self._update_statement_balances(statement_values)
+            statement = self._statement_create_or_write(statement_values)
+            self._journal_set_statement_source()
+            if not statement.line_ids:
+                statement.unlink()
+        return res
+
+
+class OFXInstitutionLine(models.Model):
+    _name = "ofx.institution.line"
+    _description = "OFX Institution Line"
+    _rec_name = "institution_id"
+
+    institution_id = fields.Many2one("ofx.institution", "Institution")
+    username = fields.Char()
+    password = fields.Char()
+    bankid = fields.Char()
+    provider_id = fields.Many2one("online.bank.statement.provider")
+    account_id = fields.Char()

account_statement_import_online_ofx/readme/CONFIGURE.rst

@@ -0,0 +1,11 @@
+To configure online bank statements provider:
+
+#. Go to *Invoicing > Configuration > Online Bank Statement Providers*
+#. Create a provider and configure provider-specific settings.
+
+If you want to allow empty bank statements to be created every time the
+information is pulled, you can check the option "Allow empty statements"
+at the provider configuration level.
+
+**NOTE**: To access these features, user needs to belong to
+*Show Full Accounting Features* group.

account_statement_import_online_ofx/readme/CONTRIBUTORS.rst

@@ -0,0 +1,3 @@
+* `ForgeFlow <https://www.forgeflow.com/>`__
+
+  * Jasmin Solanki <jasmin.solanki@forgeflow.com>

account_statement_import_online_ofx/readme/DESCRIPTION.rst

@@ -0,0 +1,3 @@
+This module provides online bank statements from Open Financial Exchange (OFX) institutions.
+You can set-up your own provider, or import a list of supported providers.
+https://ofxhome.com/ is used as a data source, currently over 300 institutions  are supported.

account_statement_import_online_ofx/readme/USAGE.rst

@@ -0,0 +1,6 @@
+To pull historical bank statements:
+
+#. Go to *Invoicing > Configuration > Journals*
+#. Select specific bank accounts
+#. Launch *Actions > Online Bank Statements Pull Wizard*
+#. Configure date interval and click *Pull*

account_statement_import_online_ofx/security/ir.model.access.csv

@@ -0,0 +1,5 @@
+id,name,model_id:id,group_id:id,perm_read,perm_write,perm_create,perm_unlink
+access_ofx_institution_manager,ofx_institution,model_ofx_institution,account.group_account_user,1,1,1,1
+access_ofx_institution_user,ofx_institution,model_ofx_institution,base.group_user,1,0,0,0
+access_ofx_institution_line_user,access_ofx_institution_line,model_ofx_institution_line,base.group_user,1,0,0,0
+access_ofx_institution_line_manager,access_ofx_institution_line,model_ofx_institution_line,account.group_account_user,1,1,1,1