Miscellaneous improvements (#27)

* RD: Added device-view.html for viewing and editing device details. Fixed and improved devices.html. Modified DeviceManagementController so that the '/monitor/device' endpoint returns the devices of the current user, and also plain users can retrieve info for the devices they own. * RD: Moved 'metrics' section side-by-side to 'device info' section in device-view.html, for better viewing * RD: changed mongo db name to 'resource_discovery' in application.yml * RD: Minor GUI improvements * RD: Added DeviceMonitorService to watch the status updates from EMS. Added DeviceStatusUpdate class used in DeviceMonitorService, and added field statusUpdate in Device class. Restructured and improved ResourceDiscoveryProperties. * RD: Created AbstractMonitorService (to factor out common topic monitoring code). Renamed DeviceMonitorService to DeviceStatusMonitorService and made it subclass of AbstractMonitorService. Changed DeviceStatusUpdate class to ignore unknown properties during deserialization. * RD: Added banner * RD: Added DeviceMetricsMonitorService and DeviceMetrics classes. Updated Device class to store device metrics received from EMS * RD: Updated application.yml * RD: Added device metrics display in device-view.html. Added (EMS) device status in devices.html * RD: Updated device-view.html to display device metrics * RD: Modified Device in monitoring subsystem to not have its credentials serialized (and returned to browser). Also updated the affected code * RD: Added check for a request's device IP address being in use (in another request or a registered device) * RD: various minor code improvements * RD: Added better error reporting in request-edit.html * RD: Various small GUI improvements and a fix * RD: Fixed registration Device in order toString() not to print credentials. Fixed RegistrationRequest in order to have its 'messages' and 'history' fields copied by BeanUtils. Fixed 2 bugs in RegistrationRequestProcessor and added a few more logs. * RD: Minor change in device-view.html to move 'count-' metrics to the end of the device metrics list * RD: Changed monitoring Device to not serialize credentials * RD: Code cleanup in RegistrationRequestProcessor * RD: Improved GUI (added page shortcuts at top-right) * RD: Implementing re-onboarding, off-boarding, and request-for-info (device status and metrics) [WIP] * RD: Added DeviceLifeCycleRequestService and DeviceLifeCycleResponseService classes for handling reinstall and uninstall actions (via EMS broker). Moved LC request event sending from DeviceManagementService to DeviceLifeCycleRequestService. Updated ResourceDiscoveryProperties with settings for DeviceLifeCycleRequest/ResponseService's. * RD: Added device archiving and unarchiving in GUI * RD: Implemented DeviceProcessor to periodically archive off-boarded devices. Added configurable immediate archiving of Success registration requests, and off-boarded devices, and added relevant settings in ResourceDiscoveryProperties and in GUI. * RD: Changed devices.html to always display 'Archive' button to Admin users * RD: Added archived-device-view.html, renamed archived-view.html to archived-request-view.html. Updated archived.html * RD: Fixed DeviceManagementController to allow plain users (device owners) to access their device info (and archived devices), and control re-installing and off-boarding. Added device colouring based on their status. * RD: Extended DeviceProcessor to check for suspect and failed devices. Added SUSPECT and FAILED statues in DeviceStatus (and in devices.html and archived.html for colouring). Added relevant settings in ResourceDiscoveryProperties. Fixed a few naming errors. * RD: Added shortcuts (top-right corner) to all detail pages * RD: Minor change in index.html (grayed out settings image) * RD: Improved AbstractMonitorService (reuse of single connection, better logs). Updated affected code. * RD: Minor code (mostly logging) and GUI improvements * RD: Added 'UnknownDeviceRegistrationService' class to monitor for unknown devices (based on IP address and reference in messages received from EMS), and registering them to RD. Added NODE_DETAILS in REQUEST_TYPE enum for messages sent to/received from EMS about acquiring detailed node info (including credentials). Added 'deviceInfoRequestsTopic' and 'deviceInfoReportsTopic' settings in ResourceDiscoveryProperties. * RD: Added prompting admin to provide device credentials when un-archiving a previously archived request or device. * RD: Moved REQUEST_TYPE enum to 'common' package and updated code * RD: Changed authentication to use BCrypt-encrypted passwords * RD: Moved message broker communication code to BrokerUtil class, and updated using classes accordingly. Implemented EncryptionUtil for encrypting and decrypting broker messages [TO BE TESTED]. Few minor code improvements. * RD: Minor code tidy up * RD: Added support for SSL connections to ActiveMQ broker in BrokerUtil. Added the needed settings in ResourceDiscoveryProperties and application.yml * RD: Fixed issue where UnknownDeviceRegistrationService registers a device before RegistrationRequestProcessor does * RD: Modified pom.xml to build an image named 'eu.nebulous.resource-discovery' and tagged with current project version * RD: Added 'DeviceLocation' class and 'location' field in both 'Device' classes. Also updated all 4 details pages to include Location (name, lat, lon). Fixed the Restore button of the modal dialog in archived.html to display 'Restore Request' or 'Restore Device' depending on the restore type. * RD: Added StatusController to report current application status (currently returns 'OK') * RD: Fixed a few minor issues: * Fixed 'spring.web.resources.static-locations' * Changed Docker image name to 'resource-discovery' * Set BPE_LANG value to C.UTF-8 in order to render banner correctly * RD: Changes in pom.xml and application.yml: * Changed groupId in pom.xml to 'eu.nebulous.resource-management', and artifactId to 'resource-discovery' * Corrected service name and description * Set 'imageName' property to derive from artifactId * Fixed application.yml to use '@project.version@' placeholder for app version * RD: Renamed 'management' module to 'resource-discovery' * RD: * Commented out Buildpacks related settings in pom.xml * Added Dockerfile, run.sh (entrypoint script) and .dockerignore * RD: Modified base images in Dockerfile * RD: Modified Dockerfile in order to both compile the app and build the Docker image * RD: Improved Dockerfile * RD: Improved Dockerfile * RD: Upgraded SB version to SB 3.2.1 and JRE to version 21. Updated dependency to their latest versions. Modified model classes to avoid Lombok SuperBuilder warnings. * RD: Fixed Dockerfile * Initial changes to support registration and deregistration of edge devices to SAL Introduction of Broker publisher and subscriber utilities Introduction of new device registration service I94a6fdb4612de192c24511445f1236cdce94b000 * EMS: Added K8sNetdataCollector [WIP] * RD: Added 'port' in forms and models * RD: Added two TODOs * RD: Deactivated UnknownDeviceRegistrationService * Initial changes to support registration and deregistration of edge devices to SAL Introduction of Broker publisher and subscriber utilities Introduction of new device registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Improvement in the initialization of SALRegistrationService I94a6fdb4612de192c24511445f1236cdce94b000 * Improvement in the initialization of SALRegistrationService I94a6fdb4612de192c24511445f1236cdce94b000 * Increased logging to debug setting of processorProperties I94a6fdb4612de192c24511445f1236cdce94b000 * Modified logging to debug the setting of processorProperties I94a6fdb4612de192c24511445f1236cdce94b000 * Correction of syntactic error I94a6fdb4612de192c24511445f1236cdce94b000 * Addition of needed configuration properties I94a6fdb4612de192c24511445f1236cdce94b000 * Testing of alternative initialization of SAL registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Testing of alternative initialization of SAL registration service I94a6fdb4612de192c24511445f1236cdce94b000 * Updates on the topic and the payload format used to communicate with SAL for the registration of a device I94a6fdb4612de192c24511445f1236cdce94b000 * Allow setting and using a custom broker port I94a6fdb4612de192c24511445f1236cdce94b000 * Pass a default port to the configuration of the Resource discovery server I94a6fdb4612de192c24511445f1236cdce94b000 * Log debugging information related to the port of the NebulOuS broker I94a6fdb4612de192c24511445f1236cdce94b000 * Publishing of the appropriate message to the broker I94a6fdb4612de192c24511445f1236cdce94b000 * RD: Removed truststore settings from application.yml. Upgraded to SB 3.2.4 and fixed pom.xml * RD: Fixed banner.txt * RD: Modified application.yml * RD: Added temp. debug messages * RD: Added temp. debug messages 2 * Revert "RD: Added temp. debug messages 2" This reverts commit 738f604. * Revert "RD: Added temp. debug messages" This reverts commit 31b103c. * RD: Modified SALRegistrationService to run SAL registration in dedicated worker thread, and interrupt registration if it takes too long. Added related settings and assigned defaults. * RD: Added temp. log messages * Revert "RD: Added temp. log messages" This reverts commit 2dae658. * RD: Updated DeviceMetricsMonitorService to match metric events to device using first the IP address and the client Id (stored in Device.StatusUpdate) * Improvements on edge device data propagation I94a6fdb4612de192c24511445f1236cdce94b000 * Integration of changes performed in commit e3bd385 but accidentally overriden I94a6fdb4612de192c24511445f1236cdce94b000 * RD: Updated Dockerfile * RD: Updated run.sh and added wait_for_mongodb.sh * RD: Improved wait_for_mongodb.sh script * Various Improvements Addition of the port attribute in the registration of an edge device Modification of the edge device registration to allow for more dynamic registration json field population I94a6fdb4612de192c24511445f1236cdce94b000 * Attempt to fix a problem when publishing to the broker to register information to SAL or notify the SLO Violation detector I94a6fdb4612de192c24511445f1236cdce94b000 * Miscellaneous improvements Updated the registration of architecture/jar files for each edge device registered Preliminary work to support 'Compromised state' * RD: Added API Key authentication * RD: Upgraded SB to 3.2.10, and Lombok, commons-lang3 dependencies to their latest versions. Improved Dockerfile. * RD: Code improvements (esp. wrt @PreAuthorize annotations) * RD: Made SALRegistrationService service conditionally enabled (by 'discovery.sal-registration.enabled' property). Its uses were updated accordingly. * RD: Updated RegistrationRequestService (and its uses) to take authenticated user into consideration during checks of requests and devices sent from GUI. Also added checks for the device data provided. * RD: Added 'Device.ref' field and added its initialization. Ref field will be populated with a unique device reference following the Nebulous naming convention (including app id). * RD: Updated GUI pages to include the new 'ref' field. A few more improvements were introduced. * Use device reference instead of name to register to SAL * Improvements in device registration and component communication with the broker * Small improvements in device registration * Small improvement in getting device registration details * Small improvement in getting device registration details * RD: Fix in SALRegistrationService class * RD: Fixed fontawesome cdn url * Initial deregistration support * RD: Made device on-boarding authorization configurable (MANUAL, ALWAYS_AUTHORIZE, and ALWAYS_REJECT) * RD: Updated RegistrationRequestProcessor to include Device Ref in onboarding request to EMS * Improvements in the handling of AMQP connections * Addition of provider field, miscellaneous improvements - Addition of provider field in device registration - Small improvements the Synchronous Broker publisher * Minor logging improvements * Deregistration process improvement * Stopping the device lost publisher * Refactoring to use the original Connector class instead of ExtendedConnector * Minor logging improvement * Only try to stop the connector if it has been previously initialized * Miscellaneous improvements Implementation of edge device/byon hourly cost (price) field Only try to stop the connector if it has been previously initialized --------- Co-authored-by: ipatini <ipatini@mail.ntua.gr> Co-authored-by: Andreas Tsagkaropoulos <atsagkaropoulos@mail.ntua.gr>
eu-nebulous · Nov 18, 2024 · 1f43782 · 1f43782
1 parent 27b0dc6
commit 1f43782
Show file tree

Hide file tree

Showing 9 changed files with 28 additions and 5 deletions.
diff --git a/...ry/src/main/java/eu/nebulous/resource/discovery/broker_communication/BrokerPublisher.java b/...ry/src/main/java/eu/nebulous/resource/discovery/broker_communication/BrokerPublisher.java
@@ -128,6 +128,16 @@ public boolean is_publisher_null(){
     public void stop(){
         if (active_connector!=null) {
             //active_connector.stop(new ArrayList<>(), publishers);
+            synchronized (active_connector_handler.getReady()){
+                while (!active_connector_handler.getReady().get()) {
+                    try {
+                        active_connector_handler.getReady().wait();
+                     } catch (InterruptedException e) {
+
+                    }
+                }
+            }
+
             active_connector.stop();
         }
     }

diff --git a/...main/java/eu/nebulous/resource/discovery/broker_communication/CustomConnectorHandler.java b/...main/java/eu/nebulous/resource/discovery/broker_communication/CustomConnectorHandler.java
@@ -14,7 +14,7 @@ public void onReady(Context context) {
         this.context = context;
         synchronized (ready) {
             this.ready.set(true);
-            this.ready.notify();
+            this.ready.notifyAll();
         }
     }
     public void remove_consumer_with_key(String key){

diff --git a/...ry/src/main/java/eu/nebulous/resource/discovery/broker_communication/SALCommunicator.java b/...ry/src/main/java/eu/nebulous/resource/discovery/broker_communication/SALCommunicator.java
@@ -148,7 +148,7 @@ public static String get_device_deregistration_json(Device device){
         return root_json_object.toJSONString();
     }
 
-    public static String get_device_registration_json(String internal_ip_address, String external_ip_address, int external_access_port, String os_family, String os_architecture, String jar_url, int os_version, int cpu_cores, long ram_gb, long disk_gb, String device_name,String provider_id, String city_name, String country_name, String device_username, String device_password, String private_key, double device_longitude, double device_latitude) {
+    public static String get_device_registration_json(String internal_ip_address, String external_ip_address, int external_access_port, String os_family, String os_architecture, String jar_url, int os_version, int cpu_cores, long ram_gb, long disk_gb, String device_name,Double price, String provider_id, String city_name, String country_name, String device_username, String device_password, String private_key, double device_longitude, double device_latitude) {
 
             JSONObject root_json_object = new JSONObject();
             JSONObject loginCredential = new JSONObject();
@@ -185,6 +185,7 @@ public static String get_device_registration_json(String internal_ip_address, St
             nodeProperties.put("numberOfCores", cpu_cores);
             nodeProperties.put("memory", ram_gb);
             nodeProperties.put("disk", disk_gb);
+            nodeProperties.put("price", price);
             nodeProperties.put("operatingSystem", operatingSystem);
             nodeProperties.put("geoLocation", geoLocation);
 

diff --git a/resource-discovery/src/main/java/eu/nebulous/resource/discovery/monitor/model/Device.java b/resource-discovery/src/main/java/eu/nebulous/resource/discovery/monitor/model/Device.java
@@ -18,6 +18,7 @@
 public class Device {
     private String id;
     private String ref;
+    private Double price;
     private String os;
     private String name;
     private String owner;

diff --git a/...java/eu/nebulous/resource/discovery/monitor/service/UnknownDeviceRegistrationService.java b/...java/eu/nebulous/resource/discovery/monitor/service/UnknownDeviceRegistrationService.java
@@ -192,6 +192,7 @@ private void processDeviceDetailsResponses() {
                 String name = map.getOrDefault("name", null).toString();
                 String owner = "-EMS-";
                 String provider = map.getOrDefault("provider", null).toString();
+                Double price = (Double) map.getOrDefault("price", null);
                 String ipAddress = map.getOrDefault("deviceIpAddress", null).toString();
                 String username = map.getOrDefault("username", null).toString();
                 char[] password = map.getOrDefault("password", "").toString().toCharArray();
@@ -237,6 +238,7 @@ private void processDeviceDetailsResponses() {
                         .name(name)
                         .owner("--EMS--")
                         .provider(provider)
+                        .price(price)
                         .requestId(requestId)
                         .ipAddress(ipAddress)
                         .nodeReference(nodeReference)

diff --git a/...rce-discovery/src/main/java/eu/nebulous/resource/discovery/registration/model/Device.java b/...rce-discovery/src/main/java/eu/nebulous/resource/discovery/registration/model/Device.java
@@ -15,6 +15,7 @@
 public class Device {
     private String id;
     private String ref;
+    private Double price;
     private String os;
     private String name;
     private String owner;

diff --git a/...in/java/eu/nebulous/resource/discovery/registration/service/SALDeregistrationService.java b/...in/java/eu/nebulous/resource/discovery/registration/service/SALDeregistrationService.java
@@ -43,7 +43,7 @@ public class SALDeregistrationService implements InitializingBean {
     private Thread processQueueThread;
     private long lastDeregistrationStartTimestamp = -1L;
 
-    public void queueForRegistration(@NonNull Device device) {
+    public void queueForDeregistration(@NonNull Device device) {
         if (processorProperties.isSalRegistrationEnabled()) //If registration is enabled, so should be deregistration as well
             queue.add(device);
     }

diff --git a/...main/java/eu/nebulous/resource/discovery/registration/service/SALRegistrationService.java b/...main/java/eu/nebulous/resource/discovery/registration/service/SALRegistrationService.java
@@ -69,13 +69,15 @@ public void register(Device device) {
         echo OS_KERNEL_RELEASE=$TMP_KERNEL_RELEASE
          */
         String device_name = device.getRef();
+        Double price = device.getPrice();
 
         int cores = Integer.parseInt(device_info.get("CPU_PROCESSORS"));
         long ram_mb = Math.round(Integer.parseInt(device_info.get("RAM_TOTAL_KB"))*1.0/1000);
         long disk_mb = Math.round(Integer.parseInt(device_info.get("DISK_TOTAL_KB"))*1.0/1000);
         String external_ip_address = device.getIpAddress();
         String device_username = device.getUsername();
-        String device_password = new String(device.getPassword());
+        String device_password = new String
+                (device.getPassword());
         String device_pub_key = new String(device.getPublicKey()); //TODO get here private key instead and pass this to device registration
         //TODO implement provider here: String provider = device.getProvider();
         //String network_rx =device_info.get("RX");
@@ -115,7 +117,7 @@ public void register(Device device) {
         //register_device_message.put("timestamp",(int)(clock.millis()/1000));
 
 
-        String register_device_message_string = get_device_registration_json(internal_ip,external_ip_address,external_access_port,os_family,os_architecture,jar_url,os_version,cores,ram_mb,disk_mb,device_name,provider_id,city_name,country_name, device_username, device_password,private_key,device_longitude, device_latitude);
+        String register_device_message_string = get_device_registration_json(internal_ip,external_ip_address,external_access_port,os_family,os_architecture,jar_url,os_version,cores,ram_mb,disk_mb,device_name,price,provider_id,city_name,country_name, device_username, device_password,private_key,device_longitude, device_latitude);
         log.info("topic is {}", get_registration_topic_name(application_name));
         log.info("broker ip is {}", processorProperties.getNebulous_broker_ip_address());
         log.info("broker port is {}", processorProperties.getNebulous_broker_port());

diff --git a/resource-discovery/src/main/resources/static/freebees_webdesign_6/request-edit.html b/resource-discovery/src/main/resources/static/freebees_webdesign_6/request-edit.html
@@ -479,6 +479,12 @@ <h5>Device details</h5>
                                 <input type="text" readonly class="form-control-plaintext" id="request#device#owner" value="" placeholder="Device Owner">
                             </div>
                         </div>
+                        <div class="form-group row">
+                            <label for="request#device#price" class="col-sm-2 col-form-label"><b>Hourly Device Cost</b></label>
+                            <div class="col-sm-10">
+                                <input type="number" name="price" min="0.00" step="0.01" class="form-control" id="request#device#price" value="0.00" placeholder="0.00">
+                            </div>
+                        </div>
                         <div class="form-group row">
                             <label for="request#device#provider" class="col-sm-2 col-form-label"><b>Device Provider</b></label>
                             <div class="col-sm-10">