Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: logging and empty config #601

Merged
merged 18 commits into from
Oct 13, 2024
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
50e2f64
license package.json
baruchiro Oct 9, 2024
0d1bcd0
enable logger
baruchiro Oct 9, 2024
62359be
Refactor logging statements to use logger instead of console.log
baruchiro Oct 9, 2024
8002010
Refactor logging statements to use logger instead of console.log
baruchiro Oct 10, 2024
cf9059d
forward main logs to renderer DevTools
baruchiro Oct 10, 2024
526ce96
fix bug when somehow the config is an empty object
baruchiro Oct 10, 2024
659052b
Merge remote-tracking branch 'origin/master' into baruch/fixes
baruchiro Oct 10, 2024
7f0c9fd
yarn was outdated on 974eca0100925e2221caa033920ab2ab42d08235
baruchiro Oct 13, 2024
32fbb8e
remove logger from yarn to pass the tests
baruchiro Oct 13, 2024
33ecde1
Merge commit '0d1bcd081bf662fe7ae5425e9f7e55cf1b1fb4ce' into temp/bisect
baruchiro Oct 13, 2024
c8c8c4d
remove what blocks the test
baruchiro Oct 13, 2024
962db48
Merge commit '62359be9daf2d4283d711bd77aa232cf9a6509fe' into temp/bisect
baruchiro Oct 13, 2024
3b43699
Merge commit '8002010509a9f2e47aafe70152f16de6ab098593' into temp/bisect
baruchiro Oct 13, 2024
fa43470
Merge commit 'cf9059d271b94b756da43e00a5d080b03986acae' into temp/bisect
baruchiro Oct 13, 2024
2263ae5
Merge commit '526ce968ccb372cad1a44f81d2040d216d5ad5cb' into temp/bisect
baruchiro Oct 13, 2024
78f0ff0
Merge commit '659052bca3a43b1d57d27f6667576b9b074d81c8' into temp/bisect
baruchiro Oct 13, 2024
29ba933
Merge commit '7f0c9fdec83fe2a0b9fbf6c6cbf68aa29b8e7e18' into temp/bisect
baruchiro Oct 13, 2024
b2e2878
Merge commit '32fbb8ea6234c780c002db3e518cea5dafce66cb' into temp/bisect
baruchiro Oct 13, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
"repository": "https://github.com/brafdlog/caspion",
"discord": "https://discord.gg/XWWg7xvJyS",
"homepage": "https://github.com/brafdlog/caspion#README.md",
"license": "MIT",
"main": "packages/main/dist/index.js",
"scripts": {
"build": "yarn build:main && yarn build:preload && yarn build:renderer",
Expand Down Expand Up @@ -75,7 +76,7 @@
"csv-parse": "^5.5.6",
"csv-stringify": "^6.5.0",
"electron-google-oauth2": "^3.1.0",
"electron-log": "^5.1.5",
"electron-log": "^5.2.0",
"electron-updater": "6.3.0",
"emittery": "^1.0.3",
"eslint-plugin-react-hooks": "^4.6.2",
Expand Down
12 changes: 9 additions & 3 deletions packages/main/src/backend/configManager/configManager.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import { type Config } from '@/backend/commonTypes';
import { decrypt, encrypt } from '@/backend/configManager/encryption/crypto';
import { existsSync, promises as fs } from 'fs';
import configExample from './defaultConfig';
import logger from '/@/logging/logger';

export async function getConfig(
configPath: string = configFilePath,
Expand All @@ -11,13 +12,18 @@ export async function getConfig(
if (configFromFile) {
const decrypted = (await decrypt(configFromFile)) as string;
if (!decrypted) {
console.log('No config file found, returning default config');
logger.log('No config file found, returning default config');
return configExample;
}
try {
return JSON.parse(decrypted);
const config = JSON.parse(decrypted);
if (Object.keys(config).length === 0) {
logger.log('Empty config file found, returning default config');
return configExample;
}
return config;
Comment on lines +19 to +24
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a fix for my case, when the config somehow became {}. (Locally on dev env)

} catch (e) {
console.error('Failed to parse config file, returning default config', e);
logger.error('Failed to parse config file, returning default config', e);
}
}

Expand Down
2 changes: 2 additions & 0 deletions packages/main/src/backend/export/exportTransactions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import {
} from '@/backend/eventEmitters/EventEmitter';
import outputVendors from '@/backend/export/outputVendors';
import _ from 'lodash';
import logger from '/@/logging/logger';

type ExecutionResult = Partial<
Record<OutputVendorName, ExportTransactionsResult>
Expand Down Expand Up @@ -62,6 +63,7 @@ export async function createTransactionsInExternalVendors(
);
executionResult[outputVendor.name] = exportTransactionsResult;
} catch (e) {
logger.error('Failed to create transactions in external vendors', e);
await eventPublisher.emit(
EventNames.EXPORTER_ERROR,
new ExporterEvent({
Expand Down
2 changes: 2 additions & 0 deletions packages/main/src/backend/export/outputVendors/csv/csv.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import { parse } from 'csv-parse/sync';
import { stringify } from 'csv-stringify/sync';
import { promises as fs } from 'fs';
import { type TransactionInstallments } from 'israeli-bank-scrapers-core/lib/transactions';
import logger from '/@/logging/logger';

export function parseTransactions(csvText: string) {
return parse(csvText, {
Expand Down Expand Up @@ -111,6 +112,7 @@ const parseTransactionsFile = async (filename: string) => {
if ((e as NodeJS.ErrnoException).code === 'ENOENT') {
return [] as EnrichedTransaction[];
}
logger.error('Failed to parse CSV file', e);
throw e;
}
};
Expand Down
2 changes: 2 additions & 0 deletions packages/main/src/backend/export/outputVendors/json/json.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import {
sortByDate,
} from '@/backend/transactions/transactions';
import { promises as fs } from 'fs';
import logger from '/@/logging/logger';

const parseTransactionsFile = async (filename: string) => {
try {
Expand All @@ -18,6 +19,7 @@ const parseTransactionsFile = async (filename: string) => {
if ((err as NodeJS.ErrnoException).code === 'ENOENT') {
return [] as EnrichedTransaction[];
}
logger.error('Failed to parse JSON file', err);
throw err;
}
};
Expand Down
2 changes: 2 additions & 0 deletions packages/main/src/backend/export/outputVendors/ynab/ynab.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ import {
import _ from 'lodash';
import moment from 'moment/moment';
import * as ynab from 'ynab';
import logger from '/@/logging/logger';

const YNAB_DATE_FORMAT = 'YYYY-MM-DD';
const NOW = moment();
Expand Down Expand Up @@ -103,6 +104,7 @@ const createTransactions: ExportTransactionsFunction = async (
allTransactions: transactionsToCreate,
}),
);
logger.error('Failed to create transactions in ynab', e);
throw e;
}
};
Expand Down
4 changes: 2 additions & 2 deletions packages/main/src/backend/import/downloadChromium.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { Browser, install } from '@puppeteer/browsers';
import logger from '/@/logging/logger';

type PuppeteerProgressCallback = (
downloadBytes: number,
Expand Down Expand Up @@ -39,8 +40,7 @@ export default async function downloadChromium(
downloadProgressCallback: progressCallback,
}).then(({ executablePath }) => {
downloadProm = null;
// TODO: eslint to use logger instead of console.log
console.log('Chromium downloaded to', executablePath);
logger.log('Chromium downloaded to', executablePath);
return executablePath;
});

Expand Down
2 changes: 2 additions & 0 deletions packages/main/src/backend/import/importTransactions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ import {
} from '../eventEmitters/EventEmitter';
import { calculateTransactionHash } from '../transactions/transactions';
import getChrome from './downloadChromium';
import logger from '/@/logging/logger';

type ScrapingConfig = Config['scraping'];

Expand Down Expand Up @@ -188,6 +189,7 @@ async function fetchTransactions(
status: AccountStatus.ERROR,
}),
);
logger.error('Failed to fetch transactions', e);
throw e;
}
}
Expand Down
2 changes: 2 additions & 0 deletions packages/main/src/backend/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import * as configManager from './configManager/configManager';
import * as Events from './eventEmitters/EventEmitter';
import outputVendors from './export/outputVendors';
import * as bankScraper from './import/bankScraper';
import logger from '../logging/logger';

export { CompanyTypes } from 'israeli-bank-scrapers-core';
export { Events, configManager, outputVendors };
Expand Down Expand Up @@ -44,6 +45,7 @@ export async function scrapeAndUpdateOutputVendors(

return executionResult;
} catch (e) {
logger.error('Failed to create transactions in external vendors', e);
await eventPublisher.emit(
Events.EventNames.GENERAL_ERROR,
new Events.BudgetTrackingEvent({
Expand Down
28 changes: 19 additions & 9 deletions packages/main/src/logging/logger.ts
Original file line number Diff line number Diff line change
@@ -1,13 +1,22 @@
import { App } from '@/app-globals';
import logger, {
type LogFunctions,
type LogLevel,
type MainErrorHandlerOptions,
} from 'electron-log'; // eslint-disable-line no-restricted-imports
import type {
LogFunctions,
LogLevel,
MainErrorHandlerOptions,
} from 'electron-log';
import log from 'electron-log/main';
import fs from 'fs';
import { EOL } from 'os';
import path from 'path';

log.initialize();

// This will transport the logs to the renderer process (DevTools) in production too
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove comment

log.transports.ipc.level = log.transports.file.level;

Object.assign(console, log.functions);
const logger = log.scope('main');
Comment on lines +12 to +18
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • initialize: Get all logs from the renderer into the main logger (the renderer can't write to file)
  • transport.ipc defaults to false for production. Since our users learned to debug the problem with the DevTools, I think it will help to send the main logs into the renderer console.
  • Object.assign(console, log.functions) to catch also the console.loga


export type Logger = LogFunctions;
export type Levels = LogLevel;

Expand All @@ -18,19 +27,20 @@ const onError: MainErrorHandlerOptions['onError'] = ({ error }) => {
logger.error(error.message || error);
if (error.stack) logger.debug(error.stack);
};
logger.errorHandler.startCatching({ onError });
logger.catchErrors({ onError });
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

deprecated

log.errorHandler.startCatching({ onError });

export const getLastLines = (n: number) => {
const lines = fs
.readFileSync(logger.transports.file.getFile().path)
.readFileSync(log.transports.file.getFile().path)
.toString()
.split(EOL);
const lastLines = lines.slice(lines.length - n);
return lastLines.join(EOL);
};

export const getLogsFolder = () =>
path.dirname(logger.transports.file.getFile().path);
path.dirname(log.transports.file.getFile().path);

logger.info(`Logs folder: ${getLogsFolder()}`);

export default logger as Logger;
5 changes: 5 additions & 0 deletions packages/renderer/src/components/App.tsx
Original file line number Diff line number Diff line change
@@ -1,9 +1,14 @@
import { useEffect } from 'react';
import { StoresProvider } from '../store';
import './App.css';
import Body from './Body';
import TopBar from './topBar/TopBar';
import logger from '../logging/logger';

function App() {
useEffect(() => {
logger.info(`${App.name} rendered`);
}, []);
return (
<StoresProvider>
<div className="App">
Expand Down
5 changes: 3 additions & 2 deletions packages/renderer/src/components/CheckForUpdates.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import {
import { useState } from 'react';
import { Button, Spinner } from 'react-bootstrap';
import { useAppInfoStore } from '../store';
import logger from '../logging/logger';

const UPDATE_STATES = {
INIT: 'INIT',
Expand Down Expand Up @@ -37,7 +38,7 @@ function CheckForUpdates() {
: UPDATE_STATES.NO_NEW_VERSION,
);
} catch (error) {
console.error(error);
logger.error(error);
setUpdateState(UPDATE_STATES.ERROR);
}
};
Expand All @@ -48,7 +49,7 @@ function CheckForUpdates() {
await downloadUpdate();
setUpdateState(UPDATE_STATES.READY_TO_INSTALL);
} catch (error) {
console.error(error);
logger.error(error);
setUpdateState(UPDATE_STATES.ERROR);
}
};
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { electronGoogleOAuth2Connector } from '#preload';
import React, { useState } from 'react';
import { Button } from 'react-bootstrap';
import { type Credentials } from '/@/types';
import logger from '/@/logging/logger';

interface LoginButtonProps {
onCredentialsChange: (credentials: Credentials) => void;
Expand All @@ -17,7 +18,7 @@ const LoginButton: React.FC<LoginButtonProps> = ({ onCredentialsChange }) => {
onCredentialsChange(credentials);
setLoading(false);
} catch (ex) {
console.error(ex);
logger.error(ex);
setLoading(false);
onCredentialsChange({});
}
Expand Down
3 changes: 3 additions & 0 deletions packages/renderer/src/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@ import { createRoot } from 'react-dom/client';
import App from './components/App';
import './index.module.css';
import reportWebVitals from './reportWebVitals';
import logger from './logging/logger';

logger.info('Frontend started');

configureMobxLinting();

Expand Down
23 changes: 23 additions & 0 deletions packages/renderer/src/logging/logger.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
import type {
LogFunctions,
LogLevel,
MainErrorHandlerOptions,
} from 'electron-log';
import log from 'electron-log/renderer';

const logger = log.scope('renderer');

export type Logger = LogFunctions;
export type Levels = LogLevel;

logger.info('Welcome to Caspion log');
logger.info('Version: <need implementation>');

const onError: MainErrorHandlerOptions['onError'] = ({ error }) => {
logger.error(error.message || error);
if (error.stack) logger.debug(error.stack);
};
log.errorHandler.startCatching({ onError });
Object.assign(console, log.functions);

export default logger as Logger;
5 changes: 3 additions & 2 deletions packages/renderer/src/store/Store.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import { getYnabAccountData } from '#preload';
import { makeAutoObservable, runInAction, toJS } from 'mobx';
import { createContext, useContext } from 'react';
import { type YnabAccountDataType, type YnabConfig } from '../types';
import logger from '../logging/logger';

// TODO: rename to YnabStore
export default class Store {
Expand All @@ -16,13 +17,13 @@ export default class Store {
}

async fetchYnabAccountData(ynabOptions: YnabConfig['options']) {
console.log('Fetching ynab account data');
logger.log('Fetching ynab account data');
this.fetchingYnabAccountData = true;
const ynabAccountData = await getYnabAccountData(ynabOptions);
runInAction(() => {
this.ynabAccountData = ynabAccountData;
this.fetchingYnabAccountData = false;
console.log('Ynab account data ', toJS(this.ynabAccountData));
logger.log('Ynab account data ', toJS(this.ynabAccountData));
});
}
}
Expand Down
8 changes: 4 additions & 4 deletions yarn.lock
Original file line number Diff line number Diff line change
Expand Up @@ -3215,10 +3215,10 @@ electron-google-oauth2@^3.1.0:
"@electron/remote" "^2.0.10"
google-auth-library "^8.8.0"

electron-log@^5.1.5:
version "5.1.5"
resolved "https://registry.yarnpkg.com/electron-log/-/electron-log-5.1.5.tgz#70d5051fc5ab7669b2592f53f72034867269c96e"
integrity sha512-vuq10faUAxRbILgQx7yHoMObKZDEfj7hMSZrJPsVrDNeCpV/HN11dU7QuY4UDUe055pzBxhSCB3m0+6D3Aktjw==
electron-log@^5.2.0:
version "5.2.0"
resolved "https://registry.yarnpkg.com/electron-log/-/electron-log-5.2.0.tgz#505716926dfcf9cb3e74f42b1003be6d865bcb88"
integrity sha512-VjLkvaLmbP3AOGOh5Fob9M8bFU0mmeSAb5G2EoTBx+kQLf2XA/0byzjsVGBTHhikbT+m1AB27NEQUv9wX9nM8w==

electron-publish@24.13.1:
version "24.13.1"
Expand Down
Loading